A recommendation model's click-through rate (CTR) has been declining for three consecutive days. The data science team has set up alerts for when CTR drops below a threshold. The alert has fired, but the team is unsure whether the decline is due to a model issue or a change in user behavior. What should be the first step in their incident response?

Investigate the model's input data for drift and check if the distribution of user features has changed.

Roll back to the previous model version.

Disable the alert to avoid noise while the team investigates.

Immediately retrain the model with the latest data.

A healthcare organization has deployed a model to predict patient readmission risk. The model's performance is monitored using AUC-ROC, which has remained stable. However, the team notices that the distribution of patient age has shifted significantly. They are concerned about the model's fairness and want to ensure it performs well across different age groups. What should they do?

Monitor performance metrics stratified by age group to detect if the model's accuracy varies across segments.

Use a different model that is less sensitive to age.

Ignore the age shift because AUC-ROC is stable.

Retrain the model with the new age distribution to ensure it adapts.

A model predicts customer churn for a telecom company. The team has set up alerts for data drift on all features. Recently, an alert fired for the 'call duration' feature, but the model's performance metrics (accuracy, precision, recall) have not changed. What is the most appropriate action?

Disable the alert for that feature to reduce noise.

Investigate the drift to determine if it is benign or if it could affect the model in the future.

Retrain the model immediately to adapt to the new distribution.

Ignore the drift because performance is stable.

CertNexusCertified Data Science Practitioner (CDSP)

Domain 6Objective 3

Monitor and Ensure Operational Performance CERTIFIED-DATA-SCIENCE-PRACTITIONER Practice Questions (Page 4)

Part of the 6.0 Operationalizing the pipeline domain, which accounts for 5-8% of the CERTIFIED-DATA-SCIENCE-PRACTITIONER exam.

30questions here

6free pages

7concepts

5-8%of the exam

Questions 16–20

16application · medium
A recommendation model's click-through rate (CTR) has been declining for three consecutive days. The data science team has set up alerts for when CTR drops below a threshold. The alert has fired, but the team is unsure whether the decline is due to a model issue or a change in user behavior. What should be the first step in their incident response?
Select an answer first
17foundation · easy
A credit card fraud detection model is monitored in production. Which metric is most appropriate to track if the business wants to minimize the number of fraudulent transactions that are missed?
Select an answer first
18expert · hard
A healthcare organization has deployed a model to predict patient readmission risk. The model's performance is monitored using AUC-ROC, which has remained stable. However, the team notices that the distribution of patient age has shifted significantly. They are concerned about the model's fairness and want to ensure it performs well across different age groups. What should they do?
Select an answer first
19expert · hard
A model predicts customer churn for a telecom company. The team has set up alerts for data drift on all features. Recently, an alert fired for the 'call duration' feature, but the model's performance metrics (accuracy, precision, recall) have not changed. What is the most appropriate action?
Select an answer first
20application · medium
A loan approval model was built using historical data where the economy was stable. After a major economic downturn, the model's accuracy has declined, but the distribution of applicant features (income, credit score) remains unchanged. Which type of drift is most likely occurring?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to CERTIFIED-DATA-SCIENCE-PRACTITIONER

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by CertNexus. “CERTIFIED-DATA-SCIENCE-PRACTITIONER” is a trademark of its owner, used for identification only.

Monitor and Ensure Operational Performance CERTIFIED-DATA-SCIENCE-PRACTITIONER Practice Questions (Page 4)

Questions 16–20

A credit card fraud detection model is monitored in production. Which metric is most appropriate to track if the business wants to minimize the number of fraudulent transactions that are missed?

A model predicts customer churn for a telecom company. The team has set up alerts for data drift on all features. Recently, an alert fired for the 'call duration' feature, but the model's performance metrics (accuracy, precision, recall) have not changed. What is the most appropriate action?

A loan approval model was built using historical data where the economy was stable. After a major economic downturn, the model's accuracy has declined, but the distribution of applicant features (income, credit score) remains unchanged. Which type of drift is most likely occurring?

Keep studying — 6.0 Operationalizing the pipeline