What is the primary purpose of comparing a shadow variant against a production variant?

To reduce the cost of inference

To replace the production variant immediately

To evaluate the new model's performance on live traffic without impacting users

To directly serve users with the new model

AWSCertified Machine Learning Engineer - Associate

Domain 2Objective 3

Task 2.3: Analyze Model Performance MLA-C01 Practice Questions (Page 3)

Part of the Content Domain 2: ML Model Development domain, which accounts for 26% of the MLA-C01 exam. AWS does not publish an official question count, but from its 130-minute exam (~50–85 total, ~13–22 in this domain), expect 4–7 from this objective — we provide 39 practice questions to prepare you well beyond it. (estimate)

39questions here

8free pages

15concepts

26%of the exam

Questions 11–15

11foundation · easy
A binary classification model produces the following confusion matrix on a test set: TP=80, TN=60, FP=20, FN=40. What is the precision of this model?
Select an answer first
12foundation · easy
What does the x-axis of a ROC curve represent?
Select an answer first
13foundation · easy
For a binary classification problem where 95% of instances belong to the negative class, what is the accuracy of a baseline model that always predicts the negative class?
Select an answer first
14foundation · easy
During model training, the training loss decreases steadily, but the validation loss starts to increase after a few epochs. What is this phenomenon called?
Select an answer first
15foundation · easy
What is the primary purpose of comparing a shadow variant against a production variant?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to MLA-C01

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by AWS. “MLA-C01” is a trademark of its owner, used for identification only.

Task 2.3: Analyze Model Performance MLA-C01 Practice Questions (Page 3)

Questions 11–15

A binary classification model produces the following confusion matrix on a test set: TP=80, TN=60, FP=20, FN=40. What is the precision of this model?

What does the x-axis of a ROC curve represent?

For a binary classification problem where 95% of instances belong to the negative class, what is the accuracy of a baseline model that always predicts the negative class?

During model training, the training loss decreases steadily, but the validation loss starts to increase after a few epochs. What is this phenomenon called?

What is the primary purpose of comparing a shadow variant against a production variant?

Keep studying — Content Domain 2: ML Model Development