Hands-On Exercise: Evaluate the Created ML Model CT-AI Practice Questions (Page 1)

Questions 1–5

1. 1application · medium

   A team evaluated a regression model predicting delivery times. The business requires that the model's predictions be within 5 minutes of the actual delivery time at least 90% of the time. Which metric would be most appropriate to assess this requirement?

   • AR-squared (R²)
   • BThe 90th percentile of the absolute errors
   • CMean Absolute Error (MAE)
   • DRoot Mean Squared Error (RMSE)
   Reveal Solution Discussion Select an answer first
2. 2foundation · easy

   A model achieves an accuracy of 95% on a dataset where 95% of instances belong to the majority class. What is the most important insight from this result?

   • AThe model is overfitting because accuracy is high.
   • BThe model has no errors because accuracy is high.
   • CThe model is performing well because accuracy is high.
   • DThe model may be biased toward the majority class and not useful for minority class detection.
   Reveal Solution Discussion Select an answer first
4. 3expert · hard

   A team evaluated a model for predicting customer churn. The business requirement is to identify at least 80% of churners (recall) while keeping the false positive rate below 20% (i.e., precision above 80%). The model currently has 85% recall and 75% precision. Which action is most appropriate?

   • ADeploy the model as-is because recall meets the requirement.
   • BRetrain the model with more data, but this is unlikely to change the trade-off.
   • CAdjust the decision threshold to increase precision, but monitor recall to ensure it stays above 80%.
   • DChange the business requirement to accept 75% precision.
   Reveal Solution Discussion Select an answer first
5. 4foundation · easy

   Which of the following elements should be included in the documentation of ML model evaluation findings?

   • AOnly the final accuracy value.
   • BThe names of the development team members only.
   • CThe dataset used, the metrics computed, and the final model performance values.
   • DThe source code of the model only.
   Reveal Solution Discussion Select an answer first
6. 5application · medium

   A team evaluated a churn prediction model against a baseline that always predicts the majority class (no churn). The model achieves 85% accuracy, while the baseline achieves 80% accuracy. The team is unsure whether the model is actually better. Which additional metric would best help compare the model to the baseline?

   • AF1-score for the churn class
   • BOverall precision across all classes
   • CThe number of training epochs
   • DThe model's training accuracy
   Reveal Solution Discussion Select an answer first