Monitor and Ensure Operational Performance CERTIFIED-DATA-SCIENCE-PRACTITIONER Practice Questions (Page 1)

Questions 1–5

1. 1foundation · easy

   Which of the following is a key component of validating a model's operational performance?

   • AVerifying that the model's output format matches what downstream systems expect
   • BTuning the model's learning rate
   • CEnsuring the model's training data is balanced
   • DSelecting the best algorithm for the problem
   Reveal Solution Discussion Select an answer first
2. 2application · medium

   A team has deployed a sentiment analysis model that processes customer reviews. They want to validate that the model is working correctly in production. Which approach best validates the model's operational performance?

   • ACheck that the model's API endpoint returns a 200 status code.
   • BEnsure the model's input schema matches the training schema.
   • CPeriodically compare the model's predictions on a sample of live reviews against human-labeled ground truth.
   • DVerify that the model's training accuracy is still above 90%.
   Reveal Solution Discussion Select an answer first
4. 3expert · hard

   A model predicts whether a customer will buy a product. The team is monitoring both data drift and concept drift. They notice that the distribution of customer age has shifted, and the model's accuracy has declined. They also notice that the relationship between age and purchase behavior has changed. What is the most appropriate action?

   • AAdjust the model's decision threshold to improve accuracy.
   • BReplace the model with a simpler model that is less sensitive to drift.
   • CRetrain the model with the latest data to adapt to both data and concept drift.
   • DOnly monitor the drift and wait for performance to stabilize.
   Reveal Solution Discussion Select an answer first
5. 4expert · hard

   A team has deployed a model that predicts inventory demand. They have set up monitoring for data quality and model performance. The model's performance metrics are stable, but the team wants to validate that the model is still appropriate for the current business environment. Which approach best validates the model's operational performance?

   • AVerify that the model's training data is still representative of the current environment.
   • BEnsure the model's code is versioned and reproducible.
   • CCheck that the model's API is responding without errors.
   • DConduct a periodic review of the model's predictions against actual outcomes and business KPIs.
   Reveal Solution Discussion Select an answer first
6. 5foundation · easy

   What is the primary use of Datadog in model monitoring?

   • ATo create dashboards and set up alerts for model performance metrics
   • BTo store training datasets
   • CTo train machine learning models
   • DTo write model code
   Reveal Solution Discussion Select an answer first