Operationalize and Monitor Models SNOWPRO-ADVANCED-DATA-SCIENTIST Practice Questions (Page 4)

Questions 16–20

1. 16foundation · easy

   What is the primary purpose of setting up alerts for model anomalies in a production environment?

   • ATo automatically fix the model without human intervention
   • BTo increase the model's prediction accuracy
   • CTo notify the appropriate team when a metric exceeds a predefined threshold
   • DTo replace the need for model monitoring dashboards
   Reveal Solution Discussion Select an answer first
2. 17expert · hard

   A bank has a credit scoring model in production. The model was trained on data from a period of economic stability. Now, during an economic downturn, the distribution of applicants' income has shifted significantly. The model's accuracy has dropped, but the bank cannot immediately retrain because labeled data for the new economic conditions is scarce. What is the most appropriate strategy to maintain model performance?

   • ARoll back to an even older model that was trained during a similar economic period.
   • BStop using the model and rely on manual review until new data is available.
   • CRetrain the model immediately using the limited new data, even if it is small.
   • DUse a drift detection method to monitor the shift and manually adjust the model's decision threshold to compensate.
   Reveal Solution Discussion Select an answer first
4. 18expert · hard

   An e-commerce company has a recommendation model in production. The model is updated weekly, and each version is stored in a model registry. After a recent update, the model's click-through rate dropped by 15% compared to the previous version. The team needs to decide whether to roll back. They have a rollback policy that requires a 24-hour observation period before reverting. What is the best course of action?

   • ARoll back immediately to the previous version to minimize revenue loss, and then investigate the cause.
   • BWait 24 hours to confirm the degradation is persistent, then roll back if the metric does not recover.
   • CKeep the new model and monitor for another week to see if the metric improves.
   • DRetrain the model immediately with the latest data and deploy the new version without rollback.
   Reveal Solution Discussion Select an answer first
5. 19application · medium

   A logistics company has a delivery time prediction model in production. The data science team wants to create an operational dashboard for both technical and non-technical stakeholders. The dashboard should show model health, usage, and performance. Which set of visualizations is most appropriate for this audience?

   • AA real-time feed of raw model predictions and errors.
   • BA dashboard with separate sections for model accuracy, data drift, prediction volume, and latency, using simple charts and clear labels.
   • CA single chart showing the model's accuracy over time, with a note explaining the metric.
   • DA series of complex technical charts showing feature importance, confusion matrices, and ROC curves.
   Reveal Solution Discussion Select an answer first
6. 20application · medium

   A telecom company has a customer churn prediction model. The model is used to target retention offers. The operations team wants to set up an alerting system that notifies them when the model's performance degrades, but they are concerned about alert fatigue. Which alerting strategy is most effective?

   • AAlert only when multiple metrics indicate a problem, and use a severity-based escalation path.
   • BAlert on every metric that falls below its threshold, regardless of severity.
   • CDisable alerts and rely on weekly manual reviews of the dashboard.
   • DAlert only when the model's accuracy drops below a very low threshold, such as 50%.
   Reveal Solution Discussion Select an answer first