Oversee AI Solution Metrics (e.g., KPI, Model Performance) PMI-CPMAI Practice Questions (Page 4)

Questions 16–20

1. 16application · medium

   A healthcare startup has deployed a model that predicts patient no-show rates. The business goal is to reduce empty appointment slots. The data science team has been tracking model accuracy, but the clinic managers are complaining that the metric does not reflect whether they are actually filling more slots. Which KPI should the team add to their monitoring to align with the business objective?

   • AThe percentage of appointment slots that were filled after the model was deployed, compared to the baseline before deployment.
   • BThe model's F1-score, because it balances precision and recall in a single number.
   • CThe number of false positives, because these represent patients who were predicted to no-show but actually attended.
   • DModel precision, because it measures how many predicted no-shows actually did not show up.
   Reveal Solution Discussion Select an answer first
2. 17application · medium

   A ride-sharing company has a surge-pricing model. The data science team wants to monitor the model's performance over time to detect if it is becoming less effective at predicting demand. They have a dashboard showing the model's mean absolute error (MAE) over the past year. What should they look for on the dashboard to identify a potential degradation trend?

   • AA single spike in MAE that occurred six months ago and then returned to normal.
   • BA steady increase in MAE over the past few months, indicating the model's predictions are becoming less accurate.
   • CA constant MAE over the past year, indicating the model is stable.
   • DA steady decrease in MAE over the past few months, indicating the model is improving.
   Reveal Solution Discussion Select an answer first
4. 18foundation · easy

   What is the primary purpose of a monitoring dashboard that displays both business KPIs and technical model metrics?

   • ATo automatically tune model hyperparameters
   • BTo provide a unified, real-time view of model health and business impact
   • CTo store historical data for model retraining
   • DTo replace the need for periodic performance reports
   Reveal Solution Discussion Select an answer first
5. 19foundation · easy

   Which of the following is a typical technical model metric that would appear on a monitoring dashboard?

   • ARevenue generated by the model
   • BModel inference latency
   • CCustomer satisfaction score
   • DNumber of active users
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   Which of the following is an example of a success measure for a customer churn prediction model?

   • AThe percentage of at-risk customers correctly identified
   • BThe number of data scientists on the team
   • CThe model's training time in minutes
   • DThe cost of cloud infrastructure
   Reveal Solution Discussion Select an answer first