Automated Retraining MACHINE-LEARNING-PROFESSIONAL Practice Questions (Page 3)

Questions 11–15

1. 11foundation · easy

   A data scientist has trained three candidate models and needs to select one for production. Which approach is most appropriate for comparing them?

   • ASelect the model with the lowest training loss
   • BPick the model with the fewest parameters
   • CChoose the model that was trained most recently
   • DEvaluate each model on a common validation dataset and compare their performance metrics
   Reveal Solution Discussion Select an answer first
2. 12foundation · easy

   Why is versioning important for models stored in a model registry?

   • AIt reduces the storage space required for each model
   • BIt automatically improves the model's accuracy
   • CIt encrypts the model artifacts for security
   • DIt enables tracking of model changes and facilitates rollback to previous versions if needed
   Reveal Solution Discussion Select an answer first
4. 13foundation · easy

   Which of the following is a valid trigger for an automated retraining workflow in a production ML system?

   • AA significant drop in the model's F1 score on live data
   • BAn increase in the number of rows in the training dataset
   • CA change in the color scheme of the monitoring dashboard
   • DA decrease in the cost of cloud compute resources
   Reveal Solution Discussion Select an answer first
5. 14application · medium

   A bank has a champion model in production for fraud detection. After a retraining run, a challenger model is produced. The team wants to evaluate the challenger against the champion before deciding to deploy it. They have a dataset of recent transactions with known labels. What is the best way to implement the champion-challenger evaluation?

   • ACompare the training loss of the challenger to the champion's training loss.
   • BEvaluate the challenger on a different dataset than the champion to see which performs better.
   • CDeploy the challenger to production immediately and monitor its performance for a week.
   • DEvaluate both models on the same held-out validation dataset and compare their performance metrics (e.g., F1 score).
   Reveal Solution Discussion Select an answer first
6. 15expert · hard

   A company has an automated retraining pipeline that is triggered by a data drift alert. The pipeline trains a new model, evaluates it, and registers it in the MLflow Model Registry. However, the team notices that the pipeline sometimes registers models that are worse than the current production model. They want to ensure that only models that outperform the current production model are registered. What is the best way to implement this?

   • AUse a fixed threshold for validation metric, and register the model if it meets the threshold.
   • BRegister the new model and then run a champion-challenger evaluation in production before transitioning it to Production stage.
   • CIn the pipeline, after training, evaluate the new model on a validation set and compare its performance to the current production model's performance on the same validation set. Only register the new model if it is better.
   • DAlways register the new model and let the deployment process decide which to use.
   Reveal Solution Discussion Select an answer first