Monitoring, Testing, and Troubleshooting AI Solutions PROFESSIONAL-MACHINE-LEARNING-ENGINEER Practice Questions (Page 5)

Questions 21–25

1. 21application · medium

   A retail company has deployed a churn prediction model on Vertex AI. The model was trained on historical data where the average customer age was 42 and the average monthly transaction count was 8. After deployment, the monitoring dashboard shows the serving data now has an average customer age of 35 and an average monthly transaction count of 12. The model's prediction accuracy has not changed. What should the ML engineer do first to investigate this situation?

   • ARetrain the model immediately with the new serving data to improve accuracy.
   • BCheck the model's feature attribution scores to see which features changed importance.
   • CCompare the feature distributions of training and serving data to confirm training-serving skew.
   • DDisable the monitoring alerts because the model accuracy is stable and no action is needed.
   Reveal Solution Discussion Select an answer first
2. 22foundation · easy

   A model that previously relied heavily on 'purchase history' now relies more on 'browsing time'. This change in which features drive predictions is known as:

   • ATraining-serving skew
   • BData drift
   • CConcept drift
   • DFeature attribution drift
   Reveal Solution Discussion Select an answer first
4. 23application · medium

   A bank has deployed a Gemini-powered loan-approval assistant that generates explanations for credit decisions. The compliance team requires that the explanations are not only accurate but also free of biased or discriminatory language. What should the ML team monitor in production?

   • AMeasure the BLEU score of the explanations against a set of approved reference explanations.
   • BMonitor the input loan applications for data drift to ensure the model sees familiar data.
   • CTrack the model's response latency to ensure explanations are generated quickly.
   • DMonitor the model's output for toxic or biased language using a safety classifier.
   Reveal Solution Discussion Select an answer first
5. 24foundation · easy

   What does concept drift monitoring track?

   • AChanges in the distribution of input features
   • BChanges in feature importance scores
   • CChanges in the relationship between input features and target outcomes
   • DChanges in model prediction latency
   Reveal Solution Discussion Select an answer first
6. 25foundation · easy

   A model was trained on data where the average customer age was 35. In production, the average age of customers is now 45. This change in the input feature distribution is an example of:

   • AModel decay
   • BFeature attribution drift
   • CData drift
   • DConcept drift
   Reveal Solution Discussion Select an answer first