A team is testing an AI system that recommends the optimal temperature for a chemical reaction. The system's output is a continuous value, and the team does not have a precise expected value for each test case. They have historical data showing that successful reactions typically occur within a certain temperature range. Which oracle approach is most appropriate?

A statistical oracle that checks if the mean of all recommendations equals the mean of historical successful temperatures.

A model-based oracle that compares the recommendation to the output of a physics-based simulation.

A human oracle where a chemist evaluates each recommendation.

A heuristic oracle that checks whether the recommended temperature falls within the historically successful range.

What is the primary purpose of a test oracle in the context of testing an AI-based system?

To generate additional training data for the AI model

To determine whether the actual output of the system matches the expected output for a given test case

To measure the performance of the hardware running the AI system

To automatically fix defects found during testing

How can a surrogate model act as an oracle in AI testing?

By using rules to determine acceptable outputs

By providing a simplified model that produces approximate expected outputs for comparison

By using human experts to evaluate the outputs

By applying statistical tests to the outputs

Certified Tester AI Testing

Domain 8Objective 7

Test Oracles for AI-Based Systems CT-AI Practice Questions (Page 6)

Part of the Domain 8: Testing AI-Specific Quality Characteristics domain, which makes up ~13% of our current practice bank. ISTQB does not publish an official question count, but from its 60-minute exam (~25–40 total, ~3–5 in this domain), expect 1–1 from this objective — we provide 29 practice questions to prepare you well beyond it. (estimate)

29questions here

6free pages

10concepts

Questions 26–29

26application · medium
A team is testing an AI system that recommends the optimal temperature for a chemical reaction. The system's output is a continuous value, and the team does not have a precise expected value for each test case. They have historical data showing that successful reactions typically occur within a certain temperature range. Which oracle approach is most appropriate?
Select an answer first
27foundation · easy
Which oracle type would be most suitable for testing an AI system that classifies medical images, where exact expected outputs are difficult to define but expert radiologists are available?
Select an answer first
28foundation · easy
What is the primary purpose of a test oracle in the context of testing an AI-based system?
Select an answer first
29foundation · easy
How can a surrogate model act as an oracle in AI testing?
Select an answer first

Finished these 4 questions?

Review the revealed explanations, or continue through the curriculum.

No more pagesBack to CT-AI

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by ISTQB. “CT-AI” is a trademark of its owner, used for identification only.

Test Oracles for AI-Based Systems CT-AI Practice Questions (Page 6)

Questions 26–29

Which oracle type would be most suitable for testing an AI system that classifies medical images, where exact expected outputs are difficult to define but expert radiologists are available?

What is the primary purpose of a test oracle in the context of testing an AI-based system?

How can a surrogate model act as an oracle in AI testing?

Keep studying — Domain 8: Testing AI-Specific Quality Characteristics