Define Success Criteria for Test CERTIFIED-DATA-SCIENCE-PRACTITIONER Practice Questions (Page 3)

Questions 11–15

1. 11expert · hard

   A data science team is developing a model for a client in the insurance industry. The client has specified that the model must have a false positive rate of less than 5% and a recall of at least 90%. The team has documented these criteria. During testing, the model achieves a false positive rate of 4% and recall of 88%. What should the team do?

   • AAccept the model because the false positive rate is within the limit
   • BChange the success criteria to be less strict
   • CRetrain the model to improve recall while keeping the false positive rate below 5%
   • DAdjust the threshold to increase recall, even if it increases the false positive rate above 5%
   Reveal Solution Discussion Select an answer first
2. 12foundation · medium

   Why is it important to document success criteria for model testing?

   • ATo ensure consistent evaluation across different testing teams and stakeholders.
   • BTo reduce the time required for model training.
   • CTo increase the model's accuracy on the test set.
   • DTo eliminate the need for model validation.
   Reveal Solution Discussion Select an answer first
4. 13application · medium

   A data science team is about to start testing a model for a client. The client has specified that the model must achieve an AUC-ROC of at least 0.90. The team has documented this in the project plan. What is the primary benefit of documenting this success criterion?

   • AIt allows the team to use any metric they want during testing
   • BIt provides a clear, measurable target that all stakeholders can agree on
   • CIt is only required for external clients, not for internal projects
   • DIt guarantees the model will achieve the target
   Reveal Solution Discussion Select an answer first
5. 14application · medium

   A marketing team wants to build a model to identify customers likely to respond to a campaign. The campaign has a limited budget, so the team wants to minimize the cost of contacting customers who do not respond. Which success criterion is most aligned with this business objective?

   • AMaximize accuracy
   • BMaximize F1-score
   • CMaximize recall
   • DMaximize precision
   Reveal Solution Discussion Select an answer first
6. 15application · medium

   A data science team is building a regression model to predict energy consumption of buildings. The stakeholders want to know how well the model explains the variability in energy consumption. Which metric should be the primary success criterion?

   • AR-squared (R²)
   • BRoot Mean Squared Error (RMSE)
   • CMean Absolute Error (MAE)
   • DAccuracy
   Reveal Solution Discussion Select an answer first