Automated Retraining MACHINE-LEARNING-PROFESSIONAL Practice Questions (Page 1)

Questions 1–5

1. 1foundation · easy

   A machine learning engineer wants to automatically retrain a model whenever the distribution of incoming feature values deviates from the training data distribution. Which mechanism should they configure as the retraining trigger?

   • AManual retraining triggered by a data scientist
   • BData drift detection
   • CModel performance degradation alerts
   • DScheduled retraining on a fixed cadence
   Reveal Solution Discussion Select an answer first
2. 2foundation · easy

   Which signal is most appropriate for triggering automated retraining based on performance degradation?

   • AA single anomalous prediction that is later corrected
   • BA sustained decline in the model's AUC on recent live data
   • CAn increase in the number of features in the dataset
   • DA change in the model's deployment environment
   Reveal Solution Discussion Select an answer first
4. 3expert · hard · select all that apply

   A company is designing an automated retraining pipeline. The pipeline should be triggered by a performance degradation alert, train a new model, evaluate it, and register it in the MLflow Model Registry. Which of the following are best practices for this pipeline? Select all that apply.

   • AEvaluate the trained model on a validation set before registering it.
   • BSkip the evaluation step to save time and register the model directly.
   • CUse a fixed schedule to trigger the pipeline, ignoring the performance alert.
   • DRegister the model in the MLflow Model Registry with appropriate versioning and stage transitions.
   • EUse a Databricks job to orchestrate the pipeline, with the trigger being the performance degradation alert.
   Check answers Discussion Select an answer first
5. 4application · medium

   A media company has an automated retraining pipeline that is triggered by a performance degradation alert. The pipeline trains a new model, evaluates it, and then registers the model in the MLflow Model Registry. The team wants to ensure that only models that pass a validation threshold are registered. What is the best way to implement this?

   • AHave the monitoring job register the model directly after training, without a separate evaluation step.
   • BRegister every trained model in the registry and let the deployment process decide which to use.
   • CIn the retraining pipeline, after training, evaluate the model on a validation set. If the metric meets the threshold, register the model in the registry; otherwise, discard it.
   • DUse the MLflow Registry API to automatically register the model without any validation.
   Reveal Solution Discussion Select an answer first
6. 5application · medium

   A manufacturing company runs automated retraining for a predictive maintenance model. After each retraining run, the pipeline registers the model in the MLflow Model Registry. The team wants to track which model version is currently in production and be able to roll back to a previous version if needed. What is the best practice?

   • AUse MLflow Model Registry stages (e.g., Staging, Production) to manage the lifecycle of model versions, and transition the selected model to Production after validation.
   • BDelete the previous model version from the registry when a new one is registered.
   • CStore model files in a shared directory and overwrite the previous version each time.
   • DKeep all model versions in the registry but do not use stages; just note the production version in a comment.
   Reveal Solution Discussion Select an answer first