Hands-On Exercise: Identify Training and Test Data and Create an ML Model CT-AI Practice Questions (Page 5)

Questions 21–23

1. 21expert · medium

   A data scientist is building a model to predict customer churn. She has a dataset with 50 features. She trains a logistic regression model and achieves 80% accuracy on the test set. She then adds a feature that is a random number generated for each customer and retrains the model. The test accuracy remains 80%. Which of the following is the most likely reason the random feature did not improve the model?

   • AThe logistic regression model is too simple to use more than 50 features.
   • BThe model is already overfitting, so adding features cannot help.
   • CThe random feature has no predictive relationship with the target variable.
   • DThe test set is too small to detect the improvement.
   Reveal Solution Discussion Select an answer first
2. 22application · medium

   An ML engineer is building a spam classifier. She has a dataset of 5,000 emails, of which 4,500 are 'not spam' and 500 are 'spam'. She trains a model and evaluates it on a test set that reflects this same class distribution. The model achieves 90% accuracy on the test set. Which of the following is the most important additional metric to examine before concluding the model is good?

   • AThe accuracy on the training set
   • BThe number of features used in the model
   • CPrecision and recall for the 'spam' class
   • DThe model's training time
   Reveal Solution Discussion Select an answer first
3. 23application · medium

   A team is developing a model to detect fraudulent credit card transactions. Fraud is rare, occurring in only 1% of transactions. They train a model and evaluate it on a test set with the same 1% fraud rate. The model's recall for the fraud class is 0.95, and its precision for the fraud class is 0.10. Which of the following is the most accurate interpretation of these metrics?

   • AThe model catches most fraud, but a large proportion of its fraud alerts are false positives.
   • BThe model has a high false-negative rate for the fraud class.
   • CThe model is useless because it only correctly predicts fraud 10% of the time.
   • DThe model is excellent because it correctly identifies 95% of all transactions.
   Reveal Solution Discussion Select an answer first