AI Assurance, Testing, and Auditing CRAGE Practice Questions (Page 8)

Questions 36–40

1. 36application · medium

   A telecom company has deployed a churn prediction model. The model's performance is monitored monthly, but the company wants to detect issues more quickly. Which change to the monitoring process would most improve early detection of model degradation?

   • AIncrease the frequency of monitoring from monthly to daily or real-time.
   • BAdd more features to the model to improve its predictive power.
   • CRetrain the model on a larger dataset every quarter.
   • DReplace the model with a simpler one that is easier to monitor.
   Reveal Solution Discussion Select an answer first
2. 37expert · hard

   A company has deployed an AI system for fraud detection. The system is working well, but the company's board wants assurance that the system will remain reliable as fraud patterns evolve. The company has limited budget and must choose between two options: (1) a comprehensive one-time audit, or (2) a lighter continuous monitoring program. Which approach best balances assurance and cost?

   • AImplement the continuous monitoring program, as it can detect drift and performance changes over time.
   • BDo both: a full audit now and continuous monitoring later, but this exceeds the budget.
   • CRely on the model's original validation results, as they demonstrate the system's reliability.
   • DConduct the comprehensive one-time audit, as it provides a thorough snapshot of the system's current state.
   Reveal Solution Discussion Select an answer first
4. 38expert · hard

   A bank is required by regulators to explain credit decisions to customers. The model in use is a complex ensemble. The bank's audit team must verify that explanations are faithful to the model's actual behavior. Which approach best ensures fidelity of explanations?

   • AUse global feature-importance scores averaged over the entire dataset.
   • BRely on the model developer's assurance that the explanations are correct.
   • CPublish the model's source code to demonstrate transparency.
   • DUse local interpretable explanations and validate them by perturbing inputs to see if the explanation matches the model's output changes.
   Reveal Solution Discussion Select an answer first
5. 39application · medium

   A bank is deploying a credit-scoring model. The model's overall accuracy is high, but the bank's compliance team is concerned about disparate impact on a protected group. The team has access to the training data, the model's predictions, and the model's feature importances. Which testing approach would most directly reveal whether the model treats the protected group unfairly?

   • AEvaluate the model's calibration by comparing predicted probabilities to observed outcomes in the overall population.
   • BRun a robustness test by adding small perturbations to the input features and measuring prediction stability.
   • CCompute the false-positive and false-negative rates separately for the protected group and the majority group.
   • DReview the model's feature importance rankings to identify which features contribute most to predictions.
   Reveal Solution Discussion Select an answer first
6. 40application · medium

   A manufacturing company uses an AI system to detect defects on an assembly line. The quality team needs to provide evidence that the system is safe and reliable to regulators. What is the best way to achieve this?

   • ARely on the vendor's marketing materials.
   • BKeep the system's performance data confidential to avoid scrutiny.
   • CConduct a formal AI assurance process that includes testing, auditing, and documentation against recognized standards.
   • DPublish the model's accuracy metrics on the company website.
   Reveal Solution Discussion Select an answer first