Oversee AI Solution Metrics (e.g., KPI, Model Performance) PMI-CPMAI Practice Questions (Page 3)

Questions 11–15

1. 11application · medium

   A bank has deployed a credit-risk model. The compliance team requires a monthly report that demonstrates the model is not discriminating against protected groups. The data science team has been tracking overall accuracy, but the compliance team says this is insufficient. What should the data science team add to their monitoring?

   • AA report showing the number of loans approved and denied for each group, without model performance metrics.
   • BA higher overall accuracy target, since a more accurate model is inherently fairer.
   • CA report showing the model's AUC, since a higher AUC means the model is better at ranking applicants.
   • DA fairness metric, such as equalized odds, that compares model performance across protected groups.
   Reveal Solution Discussion Select an answer first
2. 12application · medium

   A subscription-based streaming service has a model that predicts which users are likely to cancel their subscription. The business goal is to reduce churn by offering targeted discounts. The data science team has been tracking the model's recall, but the marketing team wants to know if the discounts are actually reducing churn. What KPI should the team add to their monitoring?

   • AThe model's precision, since it measures how many predicted churners actually churned.
   • BThe churn rate among users who received a discount, compared to the churn rate among users who did not receive a discount.
   • CThe number of discounts offered, since more discounts should lead to less churn.
   • DThe model's F1-score, since it balances precision and recall.
   Reveal Solution Discussion Select an answer first
4. 13application · medium

   A financial services firm has a fraud-detection model in production. Over the past three months, the model's recall has been steadily declining while precision has remained stable. The data science team suspects the model is missing new fraud patterns. What is the most appropriate immediate action?

   • ARetrain the model immediately with the latest data, since declining recall indicates the model is outdated.
   • BIncrease the alerting frequency to catch the decline sooner, since the current alerts are not working.
   • CLower the decision threshold to increase recall, since this will catch more fraud cases.
   • DInvestigate the data distribution and feature drift to confirm the cause of the recall decline before deciding on a fix.
   Reveal Solution Discussion Select an answer first
5. 14foundation · easy

   Which of the following is a key element of an effective performance report for stakeholders?

   • AThe raw code used to deploy the model
   • BA detailed explanation of every algorithm used in the model
   • CA list of all data sources used for training
   • DA summary of model performance metrics and business KPIs
   Reveal Solution Discussion Select an answer first
6. 15foundation · easy

   Which of the following is an example of a threshold that might trigger an alert for a real-time recommendation model?

   • AThe number of data scientists on the team exceeding 5
   • BThe number of recommendations requested exceeding 10,000 per day
   • CThe model's training data size exceeding 1 GB
   • DModel accuracy dropping below 85%
   Reveal Solution Discussion Select an answer first