Hands-On Exercise: Evaluate the Created ML Model CT-AI Practice Questions (Page 3)

Questions 11–15

1. 11application · medium

   A team developed a regression model to predict house prices. The business requirement states that the model's predictions must be within 10% of the actual price at least 80% of the time. After evaluation, the model achieves an R² of 0.85 and a Mean Absolute Percentage Error (MAPE) of 8%. Which conclusion is most appropriate?

   • AThe model meets the requirement because the MAPE is below 10%.
   • BThe requirement cannot be verified with the given metrics; a distribution of absolute percentage errors is needed.
   • CThe model fails the requirement because R² is not high enough.
   • DThe model meets the requirement because R² of 0.85 indicates strong predictive power.
   Reveal Solution Discussion Select an answer first
2. 12application · medium

   A team evaluated a spam detection model on a test set that contains 90% legitimate emails and 10% spam. The model achieved 98% accuracy. However, the recall for spam is only 40%. What is the most likely reason for the high accuracy despite low spam recall?

   • AThe test set is too small to be reliable.
   • BThe model is overfitting to the training data.
   • CThe model has a high precision for spam, which offsets the low recall.
   • DThe model is biased toward predicting the majority class (legitimate), so it misses many spam emails.
   Reveal Solution Discussion Select an answer first
4. 13foundation · easy

   Which functional performance metric is most appropriate for evaluating a binary classification model when the cost of false negatives is much higher than the cost of false positives?

   • AF1-score
   • BPrecision
   • CRecall (sensitivity)
   • DAccuracy
   Reveal Solution Discussion Select an answer first
5. 14application · medium

   A data scientist trained a binary classifier to detect fraudulent transactions. The business requirement is to catch at least 95% of all fraud cases, even if it means flagging many legitimate transactions for manual review. The model achieved 96% recall and 70% precision on the test set. Which conclusion is most appropriate?

   • AThe model meets the recall requirement, but the low precision may lead to a high number of false positives that could overwhelm the review team.
   • BThe model fails the requirement because precision is below 95%, so it should be retrained.
   • CThe model is perfect because recall is above 95%, and no further evaluation is needed.
   • DThe model should be rejected because a recall of 96% with 70% precision indicates a data leakage problem.
   Reveal Solution Discussion Select an answer first
6. 15foundation · easy

   A model was trained to detect fraudulent transactions. The business requirement states that the model must achieve at least 90% recall. The model achieves 85% recall on the validation set. What is the appropriate conclusion?

   • AThe model is acceptable because recall is above 80%.
   • BThe model should be deployed because it is better than random guessing.
   • CThe model meets the requirement because recall is close to 90%.
   • DThe model does not meet the requirement and should be improved or retrained.
   Reveal Solution Discussion Select an answer first