Task 4.1: Monitor Model Inference MLA-C01 Practice Questions (Page 4)

Questions 16–19

1. 16foundation · easy

   Which metric indicates the percentage of inference requests that result in an error?

   • AError rate
   • BLatency
   • CThroughput
   • DPrediction drift
   Reveal Solution Discussion Select an answer first
2. 17foundation · easy

   Which method is used to detect concept drift?

   • AAnalyzing the model's training loss curves.
   • BReviewing the model's feature importance scores.
   • CComparing input data distributions using statistical tests.
   • DMonitoring the model's prediction accuracy over time using labeled data.
   Reveal Solution Discussion Select an answer first
4. 18application · medium

   A retail company deploys a model that predicts daily sales. The model was trained on historical data from the past three years. Recently, the model's predictions have become less accurate, and the data science team suspects the input data distribution has changed. They want to set up automated monitoring to detect this. Which approach should they use?

   • AUse CloudWatch to monitor the model's prediction latency and set an alarm for when latency exceeds a threshold.
   • BUse SageMaker Clarify to run an explainability report on the model's predictions.
   • CUse SageMaker Model Monitor to run a data quality monitoring job that compares the input data distribution against a baseline.
   • DUse AWS CloudTrail to monitor API calls to the model endpoint and look for unusual access patterns.
   Reveal Solution Discussion Select an answer first
5. 19application · medium

   A media company has a real-time recommendation model deployed on a SageMaker endpoint. They need to monitor the model's throughput and error rates to ensure it can handle traffic spikes. They also want to set up alerts for when error rates exceed 1%. What should they use?

   • AUse SageMaker Clarify to run bias analysis on the model's predictions.
   • BUse CloudWatch to collect and visualize endpoint metrics, and create an alarm on the error rate.
   • CUse SageMaker Model Monitor to schedule hourly data quality monitoring jobs.
   • DUse AWS X-Ray to trace requests and identify latency bottlenecks.
   Reveal Solution Discussion Select an answer first