Supervision of AI Solutions (e.g., Outputs, Impacts, and Decisions) AAIA Practice Questions (Page 6)

Questions 26–30

1. 26application · medium

   A logistics company uses an AI system to optimize delivery routes. The system sometimes recommends routes that pass through low-income neighborhoods more frequently than necessary. The company's ethics committee is concerned about potential disparate impact. What is the most appropriate action for the AI operations team to take?

   • AIgnore the concern because the system is optimizing for efficiency, not social outcomes.
   • BInvestigate the system's routing logic and data to determine if the pattern is due to bias or legitimate factors.
   • CAdjust the system's optimization weights to avoid low-income neighborhoods entirely.
   • DImmediately stop using the AI system and revert to manual route planning.
   Reveal Solution Discussion Select an answer first
2. 27application · medium

   A bank uses an AI model to approve small personal loans. The model was validated before deployment, but during the first month of production, the approval rate for applicants under age 30 drops from 40% to 12% without any change to the model or input data. The bank's AI operations team needs to determine whether this is a genuine shift in applicant behavior or an emerging model error. What is the most appropriate first step?

   • AEscalate the issue to the model development team to rewrite the model's feature engineering code.
   • BIncrease the model's decision threshold to reduce the number of approvals for applicants under age 30.
   • CImmediately retrain the model using the most recent three months of loan application data.
   • DReview the model's output distribution and feature input statistics for the affected period against the validation baseline.
   Reveal Solution Discussion Select an answer first
4. 28foundation · easy

   Which mechanism is used to detect anomalies in AI outputs over time?

   • AManual review of every single output by a human
   • BOne-time validation of the AI model before deployment
   • CPeriodic deletion of old outputs to prevent errors
   • DContinuous monitoring of output metrics against established baselines
   Reveal Solution Discussion Select an answer first
5. 29expert · hard

   A bank uses an AI system to recommend loan approvals. The system was developed by an external vendor and is updated automatically by the vendor. The bank's compliance team is concerned that the updates may change the system's decision-making in ways that violate internal policies. The bank wants to maintain oversight of the AI-driven decisions. Which approach best addresses this concern?

   • ADisable automatic updates and require the bank's IT team to manually apply all updates.
   • BRequire the vendor to provide detailed documentation of each update and conduct an internal review before the update is applied.
   • CConduct a quarterly review of the system's decisions, regardless of updates.
   • DAllow the vendor to update the system automatically, trusting that the vendor will maintain compliance.
   Reveal Solution Discussion Select an answer first
6. 30foundation · easy

   What is the purpose of logging AI output errors?

   • ATo increase the speed of AI inference
   • BTo provide a record for analysis and remediation
   • CTo hide errors from stakeholders
   • DTo automatically delete the AI system
   Reveal Solution Discussion Select an answer first