Monitor and Ensure Operational Performance CERTIFIED-DATA-SCIENCE-PRACTITIONER Practice Questions (Page 6)

Questions 26–30

1. 26application · medium

   A data science team has deployed a model that predicts equipment failure. They want to set up continuous monitoring of the pipeline, including data quality and model performance. Which set of metrics should they include in their monitoring dashboard?

   • AModel training time and resource utilization
   • BOnly data pipeline execution time and error rates
   • COnly model accuracy and precision
   • DData completeness, data freshness, model accuracy, and prediction latency
   Reveal Solution Discussion Select an answer first
2. 27application · medium

   A team has set up an alert for when a model's accuracy drops below 0.80. The alert fires, but upon investigation, they find that the accuracy drop is due to a temporary issue with the data pipeline that has already been resolved. What should the team do to prevent similar false alarms in the future?

   • AIncrease the alert threshold to 0.90 to make it less sensitive.
   • BAdjust the alert threshold to be more sensitive to sustained drops rather than temporary fluctuations.
   • CDisable the alert entirely to avoid noise.
   • DIgnore the alert and continue monitoring as before.
   Reveal Solution Discussion Select an answer first
4. 28expert · hard

   A company uses Datadog to monitor its ML models. They have set up a dashboard that shows model accuracy, data drift, and pipeline latency. They want to add an alert that notifies the team when model accuracy drops below 0.75 for more than 10 minutes. Which Datadog feature should they use?

   • ACreate a monitor with a threshold condition and a duration of 10 minutes.
   • BCreate a dashboard widget that shows the accuracy metric.
   • CExport the accuracy metric to an external tool for alerting.
   • DUse Datadog APM to trace model predictions.
   Reveal Solution Discussion Select an answer first
5. 29foundation · easy

   What is the first step in responding to a model performance issue detected by an alert?

   • ARoll back the model to a previous version
   • BIgnore the alert if it is a false positive
   • CImmediately retrain the model with new data
   • DInvestigate the alert to determine the cause of the degradation
   Reveal Solution Discussion Select an answer first
6. 30foundation · easy

   A data science team has deployed a churn prediction model into production. During operational model validation, which activity is most directly focused on ensuring the model performs correctly in its live environment?

   • ARe-running the model on the original training dataset to confirm the same accuracy
   • BComparing the model's predictions on live data against expected outcomes and verifying integration points
   • CChecking the model's code for syntax errors before deployment
   • DReviewing the model's hyperparameters to ensure they are set to optimal values
   Reveal Solution Discussion Select an answer first