Test Objectives and Acceptance Criteria CT-AI Practice Questions (Page 3)

Questions 11–15

1. 11expert · hard

   A credit scoring AI system is being tested. The regulatory team requires that the system not discriminate based on protected attributes, and the business team requires that the system maximize the number of approved loans while keeping default rates low. The test lead must define test objectives. Which objective best addresses both regulatory and business needs?

   • AVerify that the approval rate is similar across protected groups and that the default rate for approved loans is below 3%.
   • BValidate that the system's default rate is below 2% for all applicants.
   • CConfirm that the system's decisions are based on at least 20 financial variables.
   • DEnsure the system approves at least 80% of all loan applications.
   Reveal Solution Discussion Select an answer first
2. 12application · medium

   An insurance company is testing an AI system that assesses damage from photos. The acceptance criterion states: 'The system shall correctly classify at least 95% of photos as damaged or not damaged.' The test lead must evaluate the testability of this criterion. Which factor most critically affects its testability?

   • AThe availability of a labeled test set with ground truth for the photos.
   • BThe number of layers in the neural network.
   • CThe programming language used to implement the model.
   • DThe color scheme of the user interface.
   Reveal Solution Discussion Select an answer first
4. 13application · medium

   A bank is deploying an AI system that approves or rejects small business loan applications. The compliance team requires that the system's decisions be explainable to regulators, while the risk team requires that the false rejection rate for creditworthy applicants stay below 2%. The test lead must define test objectives for the upcoming test cycle. Which test objective best aligns with both stakeholder needs?

   • AVerify that the model's decisions can be traced to specific input features and that the false rejection rate for creditworthy applicants is below 2%.
   • BConfirm that the model's decisions are consistent with historical human decisions.
   • CValidate that the model's output probabilities are well-calibrated across all loan amounts.
   • DEnsure the model's overall accuracy is above 95% on the validation dataset.
   Reveal Solution Discussion Select an answer first
5. 14application · medium

   A healthcare startup is developing an AI system that triages patient messages to suggest urgency levels. The product owner states: 'The system should be accurate and not miss urgent cases.' The test lead must write acceptance criteria for the triage function. Which acceptance criterion is the most testable and unambiguous?

   • AThe system shall provide a confidence score for each triage decision.
   • BThe system shall perform better than the previous version of the triage system.
   • CThe system shall correctly identify at least 99% of urgent messages in a test set of 1,000 labeled messages, with no more than 5% of non-urgent messages flagged as urgent.
   • DThe system shall be highly accurate and rarely miss urgent cases.
   Reveal Solution Discussion Select an answer first
6. 15foundation · easy

   Which factor is most important when evaluating the testability of an acceptance criterion for an AI system?

   • AThe criterion is based on the developer's opinion of system quality
   • BThe criterion is identical to the system's marketing description
   • CThe criterion is expressed in a way that allows objective measurement using the system's inputs and outputs
   • DThe criterion is written in technical jargon that only experts understand
   Reveal Solution Discussion Select an answer first