Evaluate Test Results CERTIFIED-DATA-SCIENCE-PRACTITIONER Practice Questions (Page 3)

Questions 11–15

1. 11application · medium

   A model predicts whether a customer will buy a product. The test set has 5% buyers. The model's AUC-ROC is 0.90, but the precision-recall AUC is 0.30. What does this discrepancy suggest?

   • AThe model is overfitting because the ROC AUC is high.
   • BThe model is biased toward the majority class.
   • CThe model ranks well but has low precision for the minority class.
   • DThe test set is too small.
   Reveal Solution Discussion Select an answer first
2. 12foundation · easy

   A data science team has just completed testing a binary classification model on a held-out test set. The model's accuracy is 92%, but the confusion matrix shows that the model misclassified 85% of the positive class (the minority class). Which interpretation of these test results is most appropriate?

   • AThe model is overfitting the training data, so the test set should be expanded.
   • BThe model has poor recall for the positive class, so the model needs improvement, possibly by adjusting the decision threshold or using resampling techniques.
   • CThe model is performing well overall, and no further action is needed.
   • DThe model is underfitting, so the test results should be ignored.
   Reveal Solution Discussion Select an answer first
4. 13application · medium

   A data scientist is evaluating a model for a credit scoring task. The model's test-set AUC-ROC is 0.85. The business wants to understand the model's performance at a specific threshold where the false positive rate is 5%. Which tool should the data scientist use?

   • AThe confusion matrix
   • BThe ROC curve
   • CThe learning curve
   • DThe precision-recall curve
   Reveal Solution Discussion Select an answer first
5. 14application · medium

   A data scientist is comparing two models for a spam detection task. Model A has an F1-score of 0.85, and Model B has an F1-score of 0.82. However, Model A has a much higher false positive rate. Which conclusion is most appropriate?

   • AModel A is overfitting because its F1-score is higher.
   • BModel A is better overall because F1-score is higher.
   • CModel B may be preferable if false positives are more costly than false negatives.
   • DModel B is underfitting because its F1-score is lower.
   Reveal Solution Discussion Select an answer first
6. 15application · medium

   A regression model predicts house prices. The test set has a few houses with extremely high prices (mansions) that are outliers. The model's MAE is $20,000, and its RMSE is $80,000. Which conclusion is most appropriate?

   • AThe model is overfitting because RMSE is much larger than MAE.
   • BThe model has large errors on the outlier houses, inflating RMSE.
   • CThe model is biased toward lower-priced houses.
   • DThe model is underfitting because MAE is high.
   Reveal Solution Discussion Select an answer first