A data scientist has trained a model and evaluated it on a validation set. Before deploying, the scientist wants to ensure the model generalizes well to unseen data. Which practice is most appropriate?

Re-evaluate the model on the training set to confirm high accuracy

Deploy the model and monitor performance in production

Evaluate the model on a separate test set that was not used during training or validation

Use the validation set multiple times to tune hyperparameters and then report its performance

Certified Artificial Intelligence Practitioner (CAIP)

Domain 3Objective 4

Objective 3.4 Evaluate the Model AIP-210 Practice Questions (Page 5)

Part of the 3.0 Training and Tuning ML Systems and Models domain, which accounts for 24% of the AIP-210 exam.

37questions here

8free pages

10concepts

24%of the exam

Questions 21–25

21application · medium
A data scientist has trained a model and evaluated it on a validation set. Before deploying, the scientist wants to ensure the model generalizes well to unseen data. Which practice is most appropriate?
Select an answer first
22foundation · easy
A model achieves very high accuracy on the training data but performs poorly on the validation data. Which condition does this indicate?
Select an answer first
23foundation · easy
Which statistical test is commonly used to compare the performance of two models on the same dataset to determine if the difference is statistically significant?
Select an answer first
24application · medium
A multiclass classification model predicts three classes: A, B, and C. The confusion matrix shows that class B is often confused with class C. Which metric would be most informative to evaluate the model's performance on class B specifically?
Select an answer first
25application · medium
A churn prediction model has a precision of 0.6 and recall of 0.8. The marketing team plans to use the model to target customers with a retention offer. Each offer costs $10, and the average customer lifetime value (CLV) is $50. What is the expected net value per targeted customer?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to AIP-210

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by CertNexus. “AIP-210” is a trademark of its owner, used for identification only.

Objective 3.4 Evaluate the Model AIP-210 Practice Questions (Page 5)

Questions 21–25

A data scientist has trained a model and evaluated it on a validation set. Before deploying, the scientist wants to ensure the model generalizes well to unseen data. Which practice is most appropriate?

A model achieves very high accuracy on the training data but performs poorly on the validation data. Which condition does this indicate?

Which statistical test is commonly used to compare the performance of two models on the same dataset to determine if the difference is statistically significant?

A multiclass classification model predicts three classes: A, B, and C. The confusion matrix shows that class B is often confused with class C. Which metric would be most informative to evaluate the model's performance on class B specifically?

A churn prediction model has a precision of 0.6 and recall of 0.8. The marketing team plans to use the model to target customers with a retention offer. Each offer costs $10, and the average customer lifetime value (CLV) is $50. What is the expected net value per targeted customer?

Keep studying — 3.0 Training and Tuning ML Systems and Models