A startup is developing an AI system that generates short marketing copy for social media posts. They have a limited budget and need to test a large number of generated texts. They have a few marketing experts available but cannot afford to have them review every output. Which oracle strategy is most cost-effective while still providing meaningful validation?

Use a statistical oracle to compare the word frequency distribution of generated texts to that of human-written marketing copy.

Use a heuristic oracle that checks for banned words and minimum length, and have experts review only texts that pass the heuristic.

Have the marketing experts review every generated text.

Use a model-based oracle that compares generated texts to a pre-trained language model's output.

Why is a test oracle considered essential when testing AI-based systems?

It provides a way to distinguish correct from incorrect system behavior

It automatically generates test cases

It ensures the AI model is trained on unbiased data

It eliminates the need for test data

In which scenario is a heuristic oracle most appropriate?

When human experts are available to evaluate each output

When a statistical model of the output distribution is available

When exact expected outputs are known for every test case

When approximate rules can determine if an output is within an acceptable range

Certified Tester AI Testing

Domain 8Objective 7

Test Oracles for AI-Based Systems CT-AI Practice Questions (Page 4)

Part of the Domain 8: Testing AI-Specific Quality Characteristics domain, which makes up ~13% of our current practice bank. ISTQB does not publish an official question count, but from its 60-minute exam (~25–40 total, ~3–5 in this domain), expect 1–1 from this objective — we provide 29 practice questions to prepare you well beyond it. (estimate)

29questions here

6free pages

10concepts

Questions 16–20

16application · medium
A startup is developing an AI system that generates short marketing copy for social media posts. They have a limited budget and need to test a large number of generated texts. They have a few marketing experts available but cannot afford to have them review every output. Which oracle strategy is most cost-effective while still providing meaningful validation?
Select an answer first
17foundation · easy
Which oracle type is best suited for validating AI outputs when exact expected results are not available, but approximate rules or thresholds can be applied?
Select an answer first
18foundation · easy
Which characteristic of AI systems makes defining a precise test oracle particularly difficult?
Select an answer first
19foundation · easy
Why is a test oracle considered essential when testing AI-based systems?
Select an answer first
20foundation · easy
In which scenario is a heuristic oracle most appropriate?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to CT-AI

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by ISTQB. “CT-AI” is a trademark of its owner, used for identification only.

Test Oracles for AI-Based Systems CT-AI Practice Questions (Page 4)

Questions 16–20

Which oracle type is best suited for validating AI outputs when exact expected results are not available, but approximate rules or thresholds can be applied?

Which characteristic of AI systems makes defining a precise test oracle particularly difficult?

Why is a test oracle considered essential when testing AI-based systems?

In which scenario is a heuristic oracle most appropriate?

Keep studying — Domain 8: Testing AI-Specific Quality Characteristics