Monitoring, Testing, and Troubleshooting AI Solutions PROFESSIONAL-MACHINE-LEARNING-ENGINEER Practice Questions (Page 2)

Questions 6–10

1. 6foundation · easy

   What is the purpose of using benchmarks to evaluate a generative AI solution?

   • ATo compare the model's performance against standard tasks and datasets
   • BTo reduce the cost of running the model
   • CTo generate new training data for the model
   • DTo increase the model's inference speed
   Reveal Solution Discussion Select an answer first
2. 7foundation · easy

   Which metric is commonly used to evaluate the quality of text generated by a generative AI model?

   • AMean squared error
   • BArea under the ROC curve
   • CF1 score
   • DBLEU score
   Reveal Solution Discussion Select an answer first
4. 8foundation · easy

   What should be monitored when evaluating a generative AI solution in production?

   • AOnly the cost of running the model
   • BOnly the diversity of generated outputs
   • COutput quality, safety, and alignment with expected behavior
   • DOnly model latency and throughput
   Reveal Solution Discussion Select an answer first
5. 9foundation · easy

   What does data drift monitoring primarily track?

   • AChanges in the relationship between features and target outcomes
   • BChanges in model feature importance scores
   • CChanges in the distribution of input data over time
   • DChanges in model prediction accuracy
   Reveal Solution Discussion Select an answer first
6. 10expert · hard

   An ML team is testing a Gemini-powered translation model. They have a test set of 1,000 sentences with human reference translations. They want to evaluate both the quality of the translations and the model's robustness to adversarial inputs, such as sentences with ambiguous meanings or slang. What is the best testing strategy?

   • AOnly run adversarial prompts and manually review the outputs for quality.
   • BCompute BLEU scores on the standard test set and separately run adversarial prompts to check for quality degradation.
   • COnly compute BLEU scores on the standard test set.
   • DMeasure the model's latency on the test set and use that as a proxy for quality.
   Reveal Solution Discussion Select an answer first