Automated Retraining MACHINE-LEARNING-PROFESSIONAL Practice Questions (Page 4)

Questions 16–20

1. 16application · medium

   A logistics company runs automated retraining for its delivery time prediction model. After each retraining run, multiple model versions are produced with different hyperparameters. The team wants to automatically select the best model to deploy based on a business metric (e.g., mean absolute error on a validation set) and also consider the model's inference latency. What is the best approach?

   • ASelect the model with the lowest validation error, regardless of latency.
   • BSelect the model with the lowest latency, regardless of validation error.
   • CDefine a composite score that combines validation error and latency, and select the model with the best score.
   • DRandomly select one of the top three models based on validation error.
   Reveal Solution Discussion Select an answer first
2. 17expert · hard

   A company has a model in production that predicts equipment failure. They monitor the model's F1 score on a weekly basis. If the F1 score drops by more than 0.1 compared to the baseline, they want to automatically retrain the model. However, they also want to avoid retraining if the drop is due to a temporary anomaly in the data (e.g., a sensor malfunction). What is the best way to implement this?

   • ATrigger retraining if the F1 score drops by more than 0.1, but also require that data drift is detected.
   • BManually review the F1 score drop before triggering retraining.
   • CTrigger retraining immediately when the F1 score drops by more than 0.1.
   • DTrigger retraining only if the F1 score drop is sustained for two consecutive weeks.
   Reveal Solution Discussion Select an answer first
4. 18foundation · easy

   A data scientist wants to automatically retrain a model when the distribution of a key input feature shifts significantly from the training data. Which approach directly addresses this requirement?

   • AImplementing a data drift detection mechanism that compares live feature distributions to training distributions and triggers retraining on significant divergence
   • BMonitoring the model's prediction latency and retraining if it exceeds a threshold
   • CReviewing the model's confusion matrix manually each week
   • DSetting up a daily scheduled job that retrains the model regardless of data changes
   Reveal Solution Discussion Select an answer first
5. 19foundation · easy

   In an automated retraining pipeline, what is the role of the orchestration layer when a performance degradation alert is received?

   • AIt deletes the old model from the registry
   • BIt sends an email to the data science team for manual approval
   • CIt initiates the retraining workflow by launching the training job and managing subsequent steps
   • DIt stops all incoming traffic to the model endpoint
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   What is the primary role of workflow orchestration in an automated retraining pipeline?

   • ATo store the training data in a cloud storage bucket
   • BTo visualize the model's performance on a dashboard
   • CTo manually review each retrained model before deployment
   • DTo coordinate the sequence of steps from trigger detection through training, evaluation, and registration
   Reveal Solution Discussion Select an answer first