Oversee AI Solution Metrics (e.g., KPI, Model Performance) PMI-CPMAI Practice Questions (Page 2)

Questions 6–10

1. 6foundation · easy

   What is the primary characteristic of a well-defined KPI for an AI solution?

   • AIt is identical for all AI solutions in the organization
   • BIt is based solely on model accuracy
   • CIt is defined after the model is deployed
   • DIt is aligned with the business objective and measurable
   Reveal Solution Discussion Select an answer first
2. 7expert · medium

   A government agency has a document-classification model. The agency's leadership wants a quarterly report that shows the model's performance and its impact on processing time. The data science team has noticed that the model's accuracy has been slowly declining over the quarter, but the impact on processing time is still positive. How should the team present this in the report?

   • APresent only the positive processing-time impact, since the accuracy decline is minor and would confuse leadership.
   • BPresent the accuracy decline and the positive processing-time impact side by side, with a clear explanation of the trade-off and a plan to address the decline.
   • CPresent only the accuracy decline, since it is the most important metric, and omit the processing-time impact.
   • DDelay the report until the accuracy decline is fixed, so the report only shows positive results.
   Reveal Solution Discussion Select an answer first
4. 8expert · medium

   A bank has a loan-approval model. The data science team has been monitoring the model's AUC, which has been stable. However, the bank's risk team has noticed that the model is approving more high-risk loans than before, even though the overall AUC is unchanged. What is the most likely explanation, and what should the team do?

   • AThe model is overfitting, so the team should retrain it with more regularization.
   • BThe model's decision threshold has shifted, so the team should review the threshold and the model's calibration.
   • CThe model is underfitting, so the team should add more features.
   • DThe model is fine, and the risk team is misinterpreting the data.
   Reveal Solution Discussion Select an answer first
5. 9foundation · easy

   Which of the following is a common sign of model degradation?

   • AA decrease in the cost of running the model
   • BA steady decline in model accuracy over several weeks
   • CAn increase in the number of data scientists on the team
   • DA sudden increase in the number of predictions made
   Reveal Solution Discussion Select an answer first
6. 10foundation · easy

   What is the primary purpose of an alerting system for an AI solution?

   • ATo replace the need for regular performance monitoring
   • BTo automatically fix model issues without human intervention
   • CTo notify relevant teams when performance metrics breach predefined thresholds
   • DTo generate periodic reports for stakeholders
   Reveal Solution Discussion Select an answer first