Define Success Criteria for Test CERTIFIED-DATA-SCIENCE-PRACTITIONER Practice Questions (Page 4)

Questions 16–20

1. 16expert · hard

   A data science team is building a model to predict customer churn. The dataset is imbalanced, with only 10% of customers churning. The business wants to identify churners but also wants to avoid sending retention offers to customers who are not likely to churn (to save costs). The team is debating between using AUC-ROC and precision-recall curves. Which approach is more appropriate for setting the success criterion?

   • AUse recall only and ignore precision
   • BUse the precision-recall curve to select a threshold that balances precision and recall
   • CUse accuracy as the primary metric because it is easy to interpret
   • DUse AUC-ROC as the primary metric because it is threshold-independent
   Reveal Solution Discussion Select an answer first
2. 17foundation · medium

   What should be included in the documentation of success criteria for a model?

   • AThe names of the data scientists who developed the model.
   • BThe cost of cloud computing resources used for training.
   • CThe specific metrics, their target values, and the evaluation dataset.
   • DThe programming language used to build the model.
   Reveal Solution Discussion Select an answer first
4. 18application · medium

   A team is building a multi-class classification model to categorize customer feedback into positive, neutral, or negative sentiments. The business wants to know how often the model is correct overall. Which metric should be the primary success criterion?

   • AF1-score
   • BPrecision
   • CRecall
   • DAccuracy
   Reveal Solution Discussion Select an answer first
5. 19application · medium

   A team is building a model to predict whether a patient has a particular disease. The disease is rare, and the cost of a false negative is very high (missing a diagnosis). The team wants to set a success criterion that ensures the model rarely misses a true case. Which metric should they use?

   • AF1-score
   • BRecall
   • CPrecision
   • DAccuracy
   Reveal Solution Discussion Select an answer first
6. 20application · medium

   A data science team is developing a model for a client. The client has specified that the model must achieve an F1-score of at least 0.85 on the test set. The team has documented this requirement in the project plan. Why is documenting this success criterion important?

   • AIt ensures the testing team knows exactly what to evaluate and avoids ambiguity
   • BIt allows the team to change the criterion later if the model performs poorly
   • CIt is optional because the data science team can decide the success criteria after testing
   • DIt is only needed for regulatory compliance, not for internal projects
   Reveal Solution Discussion Select an answer first