AI Assurance, Testing, and Auditing CRAGE Practice Questions (Page 2)

Questions 6–10

1. 6foundation · easy

   Which AI testing methodology is primarily concerned with verifying that the model meets specified accuracy, precision, recall, and other quality metrics?

   • ARobustness testing
   • BFunctional testing
   • CPerformance testing
   • DSecurity testing
   Reveal Solution Discussion Select an answer first
2. 7expert · hard

   An AI audit of a medical diagnosis system reveals that the model performs well on average but has a higher error rate for a rare disease. The audit team must decide how to report this finding. What is the most appropriate action?

   • AMention the finding only in the appendix without emphasis.
   • BReport the subgroup error rate as a high-priority finding and recommend targeted testing and monitoring for the rare disease.
   • CRecommend discontinuing the model entirely.
   • DOmit the subgroup finding because the overall accuracy is high.
   Reveal Solution Discussion Select an answer first
4. 8application · medium

   A hospital uses a deep-learning model to recommend treatment plans. Regulators require that clinicians understand why a specific recommendation was made for each patient. The model currently provides only a final output. What should the AI assurance team do first?

   • AReplace the deep-learning model with a linear regression model to ensure inherent interpretability.
   • BDocument the model architecture and training data in an audit report.
   • CConduct a robustness test by varying input features to see if recommendations change.
   • DImplement a post-hoc explainability tool such as SHAP or LIME to generate per-prediction explanations.
   Reveal Solution Discussion Select an answer first
5. 9expert · hard

   A lending model shows no overall bias in approval rates, but subgroup analysis reveals that for applicants with non-traditional credit histories, the model is 20% less accurate for one demographic group. The compliance team wants to ensure fairness. What is the most appropriate action?

   • AInvestigate the subgroup performance gap and consider targeted retraining or additional features to improve accuracy for that group.
   • BIncrease the model's complexity to improve overall accuracy.
   • CAccept the model because overall approval rates are balanced.
   • DRemove all demographic features from the model to eliminate bias.
   Reveal Solution Discussion Select an answer first
6. 10foundation · easy

   Which technique is commonly used to identify and evaluate AI-related risks by systematically analyzing potential failure modes and their effects?

   • APESTLE analysis
   • BSWOT analysis
   • CRoot cause analysis
   • DFailure Mode and Effects Analysis (FMEA)
   Reveal Solution Discussion Select an answer first