Evaluate Model Performance SNOWPRO-ADVANCED-DATA-SCIENTIST Practice Questions (Page 2)

Questions 6–10

1. 6application · medium

   A data scientist is evaluating a binary classifier for a marketing campaign. The model achieves an AUC-ROC of 0.90. The business wants to send promotional emails to a limited number of customers to maximize response rate. Which interpretation of AUC-ROC is most useful for this decision?

   • AThe model has 90% accuracy.
   • BThe model has a 90% probability of predicting the positive class for any customer.
   • CThe model will correctly identify 90% of all positive customers.
   • DThe model correctly ranks a randomly chosen positive customer higher than a randomly chosen negative customer 90% of the time.
   Reveal Solution Discussion Select an answer first
2. 7foundation · easy

   Which metric is defined as the proportion of true positive predictions among all instances that the model predicted as positive?

   • ARecall
   • BAccuracy
   • CF1-score
   • DPrecision
   Reveal Solution Discussion Select an answer first
4. 8foundation · easy

   A model has a precision of 0.90 and a recall of 0.60. Which interpretation is correct?

   • AThe model incorrectly labels 40% of actual positive cases as negative.
   • BThe model has a false positive rate of 10%.
   • CThe model correctly identifies 90% of all actual positive cases.
   • DThe model's F1-score is 0.75.
   Reveal Solution Discussion Select an answer first
5. 9expert · medium

   A data scientist is building a model to predict customer churn. The dataset is imbalanced (5% churn). She uses stratified 5-fold cross-validation to evaluate a model and obtains an average F1-score of 0.72. However, when she deploys the model on a new holdout set, the F1-score drops to 0.58. Which of the following is the most likely cause of this discrepancy?

   • AStratified cross-validation is not appropriate for imbalanced data.
   • BThe holdout set is from a different time period or population, causing a distribution shift.
   • CThe model overfitted to the training folds.
   • DThe F1-score is not a reliable metric for imbalanced data.
   Reveal Solution Discussion Select an answer first
6. 10expert · medium

   A data scientist is comparing two classifiers for a credit default prediction. She uses stratified 5-fold cross-validation and records the AUC-ROC for each fold. The fold-wise AUCs for Model A are [0.78, 0.82, 0.80, 0.79, 0.81] and for Model B are [0.76, 0.80, 0.79, 0.78, 0.80]. She wants to select the best model with statistical confidence. Which approach is most appropriate?

   • ASelect Model A because its mean AUC is higher.
   • BUse the McNemar test on the predictions.
   • CPerform an unpaired t-test on the fold-wise AUCs.
   • DPerform a paired t-test on the fold-wise AUCs to determine if the difference is statistically significant.
   Reveal Solution Discussion Select an answer first