A data scientist is comparing two models for a churn prediction task. She uses 5-fold cross-validation and obtains the following average AUC-ROC scores: Model X = 0.82 ± 0.03, Model Y = 0.81 ± 0.02. The standard deviation is the variation across folds. Which conclusion is most appropriate?

Both models are equivalent, so choose the simpler one without further analysis.

Model Y is better because it has lower variance, indicating more stability.

The difference is likely not statistically significant, so further testing (e.g., paired t-test) is needed before selecting a model.

Model X is significantly better because its average AUC is higher.

A data scientist is comparing two models for a binary classification problem. She uses 10-fold cross-validation and computes the accuracy for each fold. The fold-wise accuracies for Model A are [0.80, 0.82, 0.79, 0.81, 0.83, 0.80, 0.82, 0.78, 0.81, 0.80] and for Model B are [0.78, 0.80, 0.77, 0.79, 0.81, 0.78, 0.80, 0.76, 0.79, 0.78]. She wants to determine if Model A is significantly better than Model B. Which statistical test is most appropriate?

Unpaired t-test on the fold-wise accuracies.

Chi-squared test on the confusion matrices.

McNemar's test on the predictions.

Paired t-test on the fold-wise accuracies.

SnowflakeSnowPro Advanced — Data Scientist

Domain 4Objective 2

Evaluate Model Performance SNOWPRO-ADVANCED-DATA-SCIENTIST Practice Questions (Page 1)

Part of the Model Development domain, which makes up ~19% of our current practice bank.

18questions here

4free pages

5concepts

Questions 1–5

1foundation · easy
A bank is building a model to approve credit card transactions. The cost of a false negative (allowing a fraudulent transaction) is much higher than the cost of a false positive (declining a legitimate transaction). Which metric should be prioritized when evaluating this model?
Select an answer first
2application · medium
A data scientist is comparing two models for a churn prediction task. She uses 5-fold cross-validation and obtains the following average AUC-ROC scores: Model X = 0.82 ± 0.03, Model Y = 0.81 ± 0.02. The standard deviation is the variation across folds. Which conclusion is most appropriate?
Select an answer first
3foundation · easy
A data scientist is evaluating a regression model that predicts house prices. Which metric is most appropriate for measuring the average magnitude of prediction errors in the same units as the target variable?
Select an answer first
4expert · medium
A data scientist is comparing two models for a binary classification problem. She uses 10-fold cross-validation and computes the accuracy for each fold. The fold-wise accuracies for Model A are [0.80, 0.82, 0.79, 0.81, 0.83, 0.80, 0.82, 0.78, 0.81, 0.80] and for Model B are [0.78, 0.80, 0.77, 0.79, 0.81, 0.78, 0.80, 0.76, 0.79, 0.78]. She wants to determine if Model A is significantly better than Model B. Which statistical test is most appropriate?
Select an answer first
5application · medium
A data scientist is evaluating a binary classifier for a medical diagnosis. The model has a true positive rate of 0.90 and a false positive rate of 0.05. What is the specificity of this model?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to SNOWPRO-ADVANCED-DATA-SCIENTIST

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by Snowflake. “SNOWPRO-ADVANCED-DATA-SCIENTIST” is a trademark of its owner, used for identification only.

Evaluate Model Performance SNOWPRO-ADVANCED-DATA-SCIENTIST Practice Questions (Page 1)

Questions 1–5

A bank is building a model to approve credit card transactions. The cost of a false negative (allowing a fraudulent transaction) is much higher than the cost of a false positive (declining a legitimate transaction). Which metric should be prioritized when evaluating this model?

A data scientist is comparing two models for a churn prediction task. She uses 5-fold cross-validation and obtains the following average AUC-ROC scores: Model X = 0.82 ± 0.03, Model Y = 0.81 ± 0.02. The standard deviation is the variation across folds. Which conclusion is most appropriate?

A data scientist is evaluating a regression model that predicts house prices. Which metric is most appropriate for measuring the average magnitude of prediction errors in the same units as the target variable?

A data scientist is evaluating a binary classifier for a medical diagnosis. The model has a true positive rate of 0.90 and a false positive rate of 0.05. What is the specificity of this model?

Keep studying — Model Development