AI Controls Management (e.g., Evaluation, Selection, Validation) AAIR Practice Questions (Page 3)

Questions 11–15

1. 11application · medium

   An online marketplace uses an AI model to detect fraudulent listings. The current control flags listings for manual review when the model's confidence score is below a threshold. Over the past quarter, the number of false positives has increased, straining the review team. What is the most appropriate continuous improvement action?

   • AAnalyze the false positive patterns and adjust the confidence threshold or model features to reduce them without increasing false negatives
   • BIncrease the review team's capacity to handle the higher false positive volume
   • CLower the confidence threshold to flag fewer listings for review
   • DDisable the control until the model is retrained
   Reveal Solution Discussion Select an answer first
2. 12expert · hard

   A technology company has an AI control that monitors for algorithmic bias in its hiring tool. The control is validated annually, but the company has recently expanded into new markets with different demographic profiles. The risk team is concerned that the control may not be effective in the new context. What is the most appropriate continuous improvement action?

   • ARe-validate the control using data from the new markets and update the control's thresholds if needed
   • BReplace the control with a new bias detection model without validation
   • CWait until the next annual validation cycle to assess the control's performance
   • DIncrease the frequency of the control's monitoring without changing its thresholds
   Reveal Solution Discussion Select an answer first
4. 13expert · hard

   A pharmaceutical company is evaluating two controls for its AI drug-discovery model: a control that validates the model's predictions against laboratory results, and a control that monitors for data drift in the training data. The company has a very low risk appetite for incorrect predictions but also faces tight deadlines for research milestones. Which evaluation approach best balances these competing priorities?

   • ADefer both controls until after the research milestones are met to avoid delays
   • BPrioritize the data-drift monitor because it can detect issues earlier and is less disruptive to research workflows
   • CImplement both controls but reduce the frequency of the prediction-validation control to meet deadlines
   • DPrioritize the prediction-validation control because it directly addresses the highest-impact risk, and accept potential delays in research milestones
   Reveal Solution Discussion Select an answer first
5. 14foundation · easy

   What is the purpose of a feedback loop in AI control management?

   • ATo ensure controls remain static and unchanged
   • BTo continuously improve controls based on performance and emerging risks
   • CTo eliminate the need for risk assessments
   • DTo replace the need for initial control design
   Reveal Solution Discussion Select an answer first
6. 15application · medium

   An insurance company has deployed an AI underwriting model with a control that reviews denied applications for potential bias. Six months after deployment, the control has flagged a low number of cases, but the compliance team suspects the control may be missing subtle bias patterns. What is the most appropriate next step for continuous control improvement?

   • AIncrease the frequency of the control's reviews to catch more cases
   • BDisable the control until the compliance team can investigate manually
   • CReplace the control with a more complex model without further analysis
   • DAnalyze the control's performance metrics and compare them against emerging fairness benchmarks and regulatory guidance
   Reveal Solution Discussion Select an answer first