AI Controls Management (e.g., Evaluation, Selection, Validation) AAIR Practice Questions (Page 4)

Questions 16–20

1. 16expert · hard

   A social media company has implemented a control that uses a separate AI model to detect hate speech in user posts. The control's performance is measured by precision and recall. The company's risk appetite is low for allowing hate speech (high recall needed) but also low for censoring legitimate speech (high precision needed). During validation, the control achieves high recall but low precision. What is the most appropriate validation-based decision?

   • AAccept the control because high recall satisfies the primary risk of allowing hate speech
   • BDeploy the control as-is and rely on human moderators to correct false positives
   • CReplace the control with a simpler keyword-based filter to improve precision
   • DAdjust the control's decision threshold to balance precision and recall, then re-validate to confirm both metrics meet the risk appetite
   Reveal Solution Discussion Select an answer first
2. 17application · medium

   An e-commerce company has implemented a control that automatically blocks AI-generated product descriptions containing prohibited claims. Before relying on this control, the risk team wants to validate that it works as intended. Which validation method would provide the most direct evidence of the control's effectiveness?

   • AReview the control's source code to confirm it implements the policy correctly
   • BMonitor the control in production for one month and review any false positives
   • CInterview the developers who built the control to understand its intended behavior
   • DRun a test suite with a set of known prohibited and allowed descriptions to verify the control's classification accuracy
   Reveal Solution Discussion Select an answer first
4. 18foundation · easy

   Which statement best describes how AI controls should be integrated across the AI lifecycle?

   • AControls are applied continuously from design through monitoring
   • BControls are applied only when a risk event occurs
   • CControls are applied only during the deployment phase
   • DControls are applied only during the monitoring phase
   Reveal Solution Discussion Select an answer first
5. 19application · medium

   A media company is developing an AI content recommendation system. The risk team has defined controls for content moderation, user privacy, and algorithmic transparency. Which approach best integrates these controls into the AI lifecycle?

   • ADocument the controls in a risk register but defer implementation until after launch
   • BDefine control requirements during design, implement them in development, and verify them before deployment and during monitoring
   • CAssign control implementation to the model development team without formal oversight
   • DImplement all controls after the model is fully trained to avoid interfering with development
   Reveal Solution Discussion Select an answer first
6. 20expert · hard

   A government agency is selecting controls for its AI-based benefits eligibility system. The system must be transparent to the public, and the agency has a moderate risk appetite. The agency is considering two controls: a rule-based explanation generator and a post-hoc audit committee. The rule-based generator is cheaper but may not explain all decisions accurately; the audit committee is more thorough but slower. What is the best control selection?

   • ASelect neither control because the system is not high-risk
   • BSelect the audit committee as the primary control because it provides thorough oversight, and use the rule-based generator as a supplementary tool
   • CSelect both controls and run them in parallel, but do not document which control was used for each decision
   • DSelect the rule-based generator as the primary control because it is cheaper and faster
   Reveal Solution Discussion Select an answer first