Hands-On Exercise: Identify Training and Test Data and Create an ML Model CT-AI Practice Questions (Page 2)

Questions 6–10

1. 6foundation · easy

   Which metric is most appropriate for evaluating a binary classification model when the classes are imbalanced?

   • AAccuracy
   • BR-squared
   • CF1-score
   • DMean Squared Error
   Reveal Solution Discussion Select an answer first
2. 7application · easy

   A junior data scientist is creating a model to predict house prices from features like square footage, number of bedrooms, and location. He has a dataset of 1,000 homes. He wants to evaluate the model's performance. Which of the following steps should he perform to get an unbiased estimate of how the model will perform on new, unseen homes?

   • ATrain the model on all 1,000 homes and evaluate it on the same 1,000 homes.
   • BUse only the most important feature to train the model and evaluate on the full dataset.
   • CSplit the data into a training set and a test set, train on the training set, and evaluate on the test set.
   • DTrain the model on the test set and evaluate it on the training set.
   Reveal Solution Discussion Select an answer first
4. 8expert · hard

   A team is developing a model to predict whether a website visitor will make a purchase. They have a dataset of 1 million sessions. They split the data into training (80%) and test (20%) sets. After training, they evaluate the model on the test set and get an AUC of 0.85. They then deploy the model and monitor its performance. After a month, they notice the AUC has dropped to 0.70. Which of the following is the most likely explanation for this drop?

   • AThe model is underfitting because it was trained on too much data.
   • BThe test set was too small to provide a reliable AUC estimate.
   • CThe model is overfitting the training data.
   • DThe distribution of the data has changed since the model was trained, a phenomenon known as concept drift.
   Reveal Solution Discussion Select an answer first
5. 9foundation · easy

   Which tool is commonly used to create a simple machine learning model from a dataset in Python?

   • ANumPy
   • Bscikit-learn
   • CMatplotlib
   • DPandas
   Reveal Solution Discussion Select an answer first
6. 10application · medium

   A data scientist at a retail company has a dataset of 10,000 customer transactions with purchase history and a binary 'churned' label. She splits the data into a training set (8,000 rows) and a test set (2,000 rows), trains a logistic regression model on the training set, and then evaluates it on the test set. She notices the model's accuracy is 95% on the training set but only 72% on the test set. Which of the following is the most likely explanation for this discrepancy?

   • AThe model has overfitted the training data and does not generalize well to unseen data.
   • BThe data scientist should have used a larger training set to improve generalization.
   • CThe model is underfitting because logistic regression is too simple for this data.
   • DThe test set is too small to provide a reliable estimate of model performance.
   Reveal Solution Discussion Select an answer first