A team has trained three different models for a churn prediction task. Model A has the highest accuracy (0.92), Model B has the highest F1-score (0.85), and Model C has the highest AUC (0.90). The business wants to minimize false positives because each false positive costs money in retention offers. Which model should they select?

Model A, because accuracy is the most intuitive metric.

Evaluate precision-recall trade-offs and select the model with the highest precision at the desired recall level.

Model B, because F1-score balances precision and recall, and precision directly relates to false positives.

Model C, because AUC is a robust metric for ranking.

A team is building a classifier for a medical diagnosis application. They have a small dataset and need a reliable estimate of model performance. They plan to use k-fold cross-validation. What is the most important consideration to ensure the performance estimate is reliable?

Ensuring the data is shuffled and split randomly without any stratification.

Using stratified k-fold to maintain the class distribution in each fold, especially for imbalanced data.

Using a single holdout validation set instead of cross-validation to save time.

Using a large number of folds, such as 10, to maximize the training data in each fold.

A team is using Bayesian optimization for hyperparameter tuning. They have a limited compute budget and need to decide between running more trials with fewer epochs per trial or fewer trials with more epochs per trial. What is the best strategy?

Use a fixed number of epochs for all trials, regardless of budget.

Use random search instead of Bayesian optimization to simplify the process.

Run fewer trials with more epochs to get more accurate performance estimates for each trial.

Run more trials with fewer epochs to explore more hyperparameter combinations, then fine-tune the best.

PMICertified Professional in Managing AI (PMI-CPMAI)

Domain 4Objective 3

Manage AI/ML Model Training PMI-CPMAI Practice Questions (Page 2)

Part of the Manage AI Model Development and Evaluation domain, which accounts for 16% of the PMI-CPMAI exam. PMI does not publish an official question count, but from its 160-minute exam (~65–105 total, ~10–17 in this domain), expect 2–3 from this objective — we provide 36 practice questions to prepare you well beyond it. (estimate)

36questions here

8free pages

10concepts

16%of the exam

Questions 6–10

6application · medium
A team has trained three different models for a churn prediction task. Model A has the highest accuracy (0.92), Model B has the highest F1-score (0.85), and Model C has the highest AUC (0.90). The business wants to minimize false positives because each false positive costs money in retention offers. Which model should they select?
Select an answer first
7foundation · easy
Which of the following is a hyperparameter tuning search strategy?
Select an answer first
8foundation · easy
Which of the following is a business criterion that might be considered during model selection?
Select an answer first
9application · medium
A team is building a classifier for a medical diagnosis application. They have a small dataset and need a reliable estimate of model performance. They plan to use k-fold cross-validation. What is the most important consideration to ensure the performance estimate is reliable?
Select an answer first
10expert · hard
A team is using Bayesian optimization for hyperparameter tuning. They have a limited compute budget and need to decide between running more trials with fewer epochs per trial or fewer trials with more epochs per trial. What is the best strategy?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to PMI-CPMAI

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by PMI. “PMI-CPMAI” is a trademark of its owner, used for identification only.

Manage AI/ML Model Training PMI-CPMAI Practice Questions (Page 2)

Questions 6–10

Which of the following is a hyperparameter tuning search strategy?

Which of the following is a business criterion that might be considered during model selection?

A team is building a classifier for a medical diagnosis application. They have a small dataset and need a reliable estimate of model performance. They plan to use k-fold cross-validation. What is the most important consideration to ensure the performance estimate is reliable?

A team is using Bayesian optimization for hyperparameter tuning. They have a limited compute budget and need to decide between running more trials with fewer epochs per trial or fewer trials with more epochs per trial. What is the best strategy?

Keep studying — Manage AI Model Development and Evaluation