Supervision of AI Solutions (e.g., Outputs, Impacts, and Decisions) AAIA Practice Questions (Page 4)

Questions 16–20

1. 16application · medium

   An insurance company uses an AI model to recommend claim approvals. A claims adjuster notices that the model consistently recommends denying claims from a particular postal code, even when the claim details are similar to approved claims elsewhere. The adjuster escalates this to the AI operations team. What should the team do first to ensure decision oversight aligns with organizational policies and ethical standards?

   • AImmediately disable the model and process all claims manually.
   • BAdjust the model's decision threshold to increase approvals for that postal code.
   • CReview the model's decision rules and training data for potential bias related to the postal code.
   • DDocument the adjuster's concern and continue monitoring for another month to see if the pattern persists.
   Reveal Solution Discussion Select an answer first
2. 17expert · hard

   A hospital uses an AI system to assist with patient discharge planning. The system recommends a discharge date based on the patient's condition. During a routine audit, the team discovers that the system has been recommending discharge dates that are too early for a subset of patients, leading to readmissions. The hospital's leadership wants to address this error while minimizing disruption to patient care. Which approach best balances patient safety and operational continuity?

   • AAdjust the system's discharge date algorithm to add an extra day for all patients.
   • BRetrain the model with the latest patient data and redeploy it immediately.
   • CImmediately disable the AI system and require all discharge decisions to be made by human clinicians.
   • DImplement a temporary human-in-the-loop review for all discharge recommendations while the root cause is investigated.
   Reveal Solution Discussion Select an answer first
4. 18application · medium

   An organization's AI operations team has been monitoring an AI system that recommends marketing campaign targets. Over the past quarter, the team identified and corrected several anomalies in the system's outputs. The compliance officer requests a report on the supervision activities. What should the team include in the report to best support transparency and accountability?

   • AA recommendation to replace the AI system with a different vendor's product.
   • BA summary of the anomalies detected, the actions taken to correct them, and the current status of each.
   • CA list of the team members who performed the monitoring activities.
   • DA comparison of the AI system's performance against competitors' systems.
   Reveal Solution Discussion Select an answer first
5. 19expert · hard

   A healthcare provider is deploying an AI system that triages patient messages and recommends urgency levels. The system was trained on historical data. The provider's compliance team requires that the system's impact on patient outcomes be assessed before full deployment. The AI operations team has limited resources and must balance thoroughness with speed. Which approach best satisfies the compliance requirement while managing resource constraints?

   • AConduct a full randomized controlled trial comparing AI triage to human triage over six months before any deployment.
   • BLimit the deployment to a single clinic and collect anecdotal feedback from clinicians for one month.
   • CPerform a retrospective impact assessment using historical data to simulate the AI's triage decisions and compare them to actual outcomes.
   • DDeploy the system immediately and rely on post-deployment monitoring to detect any negative impacts.
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   Which of the following is an example of overseeing AI-driven decisions?

   • AIncreasing the AI model's confidence threshold to reduce false positives
   • BDeleting all AI-generated decisions to avoid errors
   • CGranting the AI system full autonomy to make all decisions
   • DReviewing a sample of AI-generated decisions for compliance with ethical guidelines
   Reveal Solution Discussion Select an answer first