Task 2.3: Analyze Model Performance MLA-C01 Practice Questions (Page 1)

Questions 1–5

1. 1expert · hard

   A data scientist is training a deep learning model for sentiment analysis. The training loss decreases rapidly for the first 5 epochs, then starts to increase. The validation loss increases from the beginning. The data scientist uses SageMaker Debugger to monitor the training job. Which of the following is the most likely cause and the most appropriate action?

   • AThe learning rate is too high, causing the loss to diverge. Reduce the learning rate.
   • BThe data is not normalized. Apply feature scaling.
   • CThe model is underfitting. Increase the model's capacity.
   • DThe model is overfitting. Add dropout and early stopping.
   Reveal Solution Discussion Select an answer first
2. 2application · medium

   A healthcare company is developing a binary classifier to screen for a rare disease. The dataset is highly imbalanced, with only 1% positive cases. They have trained two models and evaluated them on a held-out test set. Model A has an AUC of 0.95, and Model B has an AUC of 0.90. The business needs to minimize the number of false negatives (missed disease cases) and is willing to accept a higher number of false positives for initial screening. Which statement is most accurate?

   • AAUC is not a valid metric for imbalanced datasets, so the team should use accuracy instead.
   • BModel A is preferred because a higher AUC indicates better discrimination between classes, and the threshold can be lowered to achieve high recall.
   • CModel B is preferred because a lower AUC means the model is less overfit.
   • DModel A is always better than Model B because it has a higher AUC.
   Reveal Solution Discussion Select an answer first
4. 3application · medium

   A data scientist is training a large language model on SageMaker. The training loss is decreasing steadily for the first 10 epochs, but then plateaus. The validation loss starts increasing after epoch 12. The data scientist uses SageMaker Debugger to monitor the training job. Which combination of actions is most appropriate?

   • ASwitch to a more complex model architecture to better capture patterns.
   • BReduce the learning rate and continue training for more epochs.
   • CIncrease the batch size to improve convergence and reduce overfitting.
   • DStop training and use the model from epoch 10, then apply early stopping.
   Reveal Solution Discussion Select an answer first
5. 4foundation · easy

   What is the primary purpose of establishing a baseline model before training a more complex model?

   • ATo reduce the training time of the complex model.
   • BTo provide a simple benchmark that the complex model should outperform.
   • CTo automatically tune hyperparameters.
   • DTo eliminate the need for cross-validation.
   Reveal Solution Discussion Select an answer first
6. 5foundation · easy

   A model predicts continuous values for energy consumption. Which metric is appropriate for evaluating this model?

   • AAUC
   • BF1 score
   • CPrecision
   • DMean Absolute Error (MAE)
   Reveal Solution Discussion Select an answer first