AI Audit Outputs and Reports AAIA Practice Questions (Page 4)

Questions 16–20

1. 16application · medium

   An AI audit of a medical diagnosis support system found that the model's accuracy is high, but it has a known limitation: it performs poorly on patients with rare conditions. The audit report should:

   • AClearly state the limitation and recommend that the system be used only as a decision support tool, not as a sole decision-maker.
   • BHighlight the high accuracy and omit the limitation to avoid undermining confidence in the system.
   • CRecommend that the system be used for all patients, as the overall accuracy is high.
   • DState that the limitation is acceptable because the system is only a support tool.
   Reveal Solution Discussion Select an answer first
2. 17application · medium

   An AI audit of a hiring tool found that the model's recommendations are biased against certain demographic groups. The root cause is that the training data reflects historical biases. The audit report should recommend:

   • AAdjusting the model's decision threshold to reduce bias, without changing the training data.
   • BAdding a disclaimer to the tool's output that it may be biased.
   • CDiscontinuing the use of the hiring tool immediately, as any bias is unacceptable.
   • DRebuilding the training dataset to be more representative and implementing ongoing bias monitoring.
   Reveal Solution Discussion Select an answer first
4. 18foundation · easy

   Which technique is most effective for communicating an AI-specific finding that involves model uncertainty?

   • AQuantify the uncertainty and explain its potential impact on the audit conclusion
   • BUse vague language such as 'possibly' without further explanation
   • COmit the finding if uncertainty is present
   • DState the finding as a definitive fact without mentioning uncertainty
   Reveal Solution Discussion Select an answer first
5. 19application · medium

   An AI audit of a fraud detection system found that the model has a high false-positive rate, leading to customer dissatisfaction. The root cause is a combination of imbalanced training data and overly aggressive threshold settings. The audit report should recommend:

   • ASwitching to a different machine learning algorithm that is known to have lower false-positive rates.
   • BImplementing a manual review process for all flagged transactions to reduce customer impact.
   • CAdjusting the decision threshold to balance false positives and false negatives, and rebalancing the training data.
   • DIncreasing the volume of all transactions to improve model training, without specific changes to data composition.
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   Which practice is a key component of quality assurance for AI audit reports?

   • APeer review by another qualified auditor
   • BUsing complex language to demonstrate expertise
   • CAvoiding any changes after the first draft
   • DEnsuring the report is written by a single author for consistency
   Reveal Solution Discussion Select an answer first