Evaluate Test Results CERTIFIED-DATA-SCIENCE-PRACTITIONER Practice Questions (Page 4)

Questions 16–20

1. 16application · easy

   A data science team is evaluating a model for a recommendation system. The model's test-set RMSE is 1.2, and the baseline model (always predicting the average rating) has an RMSE of 1.5. What does this indicate?

   • AThe model is underfitting because its RMSE is not zero.
   • BThe model is overfitting because its RMSE is lower than the baseline.
   • CThe model is biased because its RMSE is lower than the baseline.
   • DThe model outperforms the baseline in predicting ratings.
   Reveal Solution Discussion Select an answer first
2. 17application · easy

   A regression model predicts energy consumption. The test set has a few extreme outliers (e.g., due to sensor malfunctions). Which metric would be most robust to these outliers when reporting model performance?

   • AMean absolute error (MAE)
   • BRoot mean squared error (RMSE)
   • CMean squared error (MSE)
   • DR-squared
   Reveal Solution Discussion Select an answer first
4. 18application · medium

   A data scientist evaluates a binary classifier on a test set and obtains an AUC-ROC of 0.95. However, the business requires a high recall for the positive class (disease detection) and is willing to accept many false positives. Which threshold should the data scientist choose?

   • AA high decision threshold
   • BA low decision threshold
   • CThe threshold that minimizes the false positive rate
   • DThe threshold that maximizes accuracy
   Reveal Solution Discussion Select an answer first
5. 19application · easy

   A data science team is evaluating a model for a medical diagnosis task. The cost of a false negative (missing a disease) is much higher than a false positive. Which metric should the team optimize?

   • ARecall
   • BPrecision
   • CAccuracy
   • DF1-score
   Reveal Solution Discussion Select an answer first
6. 20application · medium

   A model trained to predict loan default has a test-set log loss of 0.65. A baseline model that always predicts the majority class has a log loss of 0.72. What does this comparison indicate?

   • AThe model is underfitting because its log loss is not zero.
   • BThe model's predictions are biased toward the majority class.
   • CThe model provides better-calibrated probability estimates than the baseline.
   • DThe model is overfitting because its log loss is lower than the baseline.
   Reveal Solution Discussion Select an answer first