A company uses Datadog to monitor its ML models. They want to set up an alert that notifies the data science team when the model's precision drops below a certain threshold. Which Datadog feature should they use?

Create a monitor on the precision metric with a threshold condition.

Use Datadog APM to trace model predictions.

Create a dashboard with the precision metric.

Export the precision metric to an external monitoring tool.

A company uses Datadog to monitor its production machine learning models. They want to create a dashboard that shows the model's performance metrics and the health of the data pipeline in one place. Which Datadog feature should they use to combine these metrics?

Export all metrics to a separate BI tool for visualization.

Use Datadog's APM traces to visualize model predictions.

Set up a monitor for each metric and send alerts to email.

Create a dashboard with widgets for model accuracy, data freshness, and pipeline latency.

A data science team has deployed a real-time recommendation model. They want to set up monitoring that can detect both data quality issues and model performance degradation. They have limited engineering resources and need to prioritize. Which approach is most effective?

Monitor only model accuracy and rely on manual checks for data quality.

Implement a lightweight monitoring solution that tracks key metrics like data freshness, feature drift, and model accuracy, with alerts for anomalies.

Implement a complex monitoring system that tracks every possible metric, including infrastructure and application logs.

Use a third-party monitoring tool that requires significant configuration but provides comprehensive features.

A model predicts equipment failure in a manufacturing plant. The team has set up an alert for when the model's false positive rate exceeds 5%. The alert has fired, and the team suspects data drift. They have limited time to respond. What should they do first?

Check the distribution of sensor readings against the training data to confirm data drift.

Roll back to the previous model version.

Increase the alert threshold to 10% to reduce false alarms.

Retrain the model with the latest sensor data.

A model predicts whether a customer will default on a loan. The model was trained on data from a period of low interest rates. Now interest rates have risen, and the model's recall has dropped significantly, while precision has improved. The distribution of customer features (income, credit score) has not changed. What is the most likely explanation?

The model is overfitting to the training data.

The model's decision threshold has shifted.

Concept drift has occurred because the relationship between features and default risk has changed.

Data drift has occurred in the target variable.

CertNexusCertified Data Science Practitioner (CDSP)

Domain 6Objective 3

Monitor and Ensure Operational Performance CERTIFIED-DATA-SCIENCE-PRACTITIONER Practice Questions (Page 5)

Part of the 6.0 Operationalizing the pipeline domain, which accounts for 5-8% of the CERTIFIED-DATA-SCIENCE-PRACTITIONER exam.

30questions here

6free pages

7concepts

5-8%of the exam

Questions 21–25

21application · medium
A company uses Datadog to monitor its ML models. They want to set up an alert that notifies the data science team when the model's precision drops below a certain threshold. Which Datadog feature should they use?
Select an answer first
22application · medium
A company uses Datadog to monitor its production machine learning models. They want to create a dashboard that shows the model's performance metrics and the health of the data pipeline in one place. Which Datadog feature should they use to combine these metrics?
Select an answer first
23expert · hard
A data science team has deployed a real-time recommendation model. They want to set up monitoring that can detect both data quality issues and model performance degradation. They have limited engineering resources and need to prioritize. Which approach is most effective?
Select an answer first
24expert · hard
A model predicts equipment failure in a manufacturing plant. The team has set up an alert for when the model's false positive rate exceeds 5%. The alert has fired, and the team suspects data drift. They have limited time to respond. What should they do first?
Select an answer first
25expert · hard
A model predicts whether a customer will default on a loan. The model was trained on data from a period of low interest rates. Now interest rates have risen, and the model's recall has dropped significantly, while precision has improved. The distribution of customer features (income, credit score) has not changed. What is the most likely explanation?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to CERTIFIED-DATA-SCIENCE-PRACTITIONER

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by CertNexus. “CERTIFIED-DATA-SCIENCE-PRACTITIONER” is a trademark of its owner, used for identification only.