Monitoring, Testing, and Troubleshooting AI Solutions PROFESSIONAL-MACHINE-LEARNING-ENGINEER Practice Questions (Page 7)

Questions 31–33

1. 31foundation · easy

   What does feature attribution drift monitoring compare over time?

   • AFeature distributions between training and serving data
   • BFeature importance scores
   • CModel prediction accuracy
   • DInput data distributions
   Reveal Solution Discussion Select an answer first
2. 32application · medium

   An ML engineer is testing a new summarization model before production deployment. The team is concerned about the model producing factually incorrect summaries when given adversarial or ambiguous inputs. Which testing strategy should the engineer use to proactively identify these failure modes?

   • ATest the model with a curated set of adversarial prompts designed to induce hallucinations or contradictions.
   • BDeploy the model to production and rely on user feedback to identify errors.
   • CRun a batch of typical customer documents and measure the ROUGE score against reference summaries.
   • DMonitor the model's latency and throughput under peak load to ensure it meets SLAs.
   Reveal Solution Discussion Select an answer first
3. 33application · medium

   A company has deployed a customer-support chatbot powered by a Gemini model. The ML team wants to monitor whether the chatbot's responses are safe, non-toxic, and aligned with the company's brand voice. Which monitoring approach should they implement?

   • ACompare the chatbot's responses to the training data using BLEU scores.
   • BSet up automated evaluation of sampled responses using safety classifiers and alignment metrics.
   • CMonitor the input prompt distribution for data drift to ensure the model sees familiar inputs.
   • DTrack the model's token-level perplexity scores on live traffic to detect quality degradation.
   Reveal Solution Discussion Select an answer first