What does a completeness metric for generated test cases typically measure?

The number of test cases that fail when executed

The consistency of the test case format

The degree to which requirements or edge cases are covered

The time taken to generate the test cases

When evaluating a generated defect report, which metric is most important to assess its usefulness for developers?

The accuracy of the steps to reproduce the defect

The consistency of the report's formatting with previous reports

The number of words in the report

The time taken to generate the report

A QA team uses a generative AI tool to create test cases for a new payment module. They want to ensure the generated suite exercises all the acceptance criteria in the user stories, including boundary values for transaction amounts. Which metric should they prioritize to evaluate the generated test cases?

Requirement coverage percentage

Semantic similarity to existing test cases

Average response time per test case

A team uses a generative AI service to create test data for a large-scale load test. The relevance score is high (90%), but the cost per record is high, and the generation time is long. The team has a limited budget and a tight deadline. What should they prioritize?

Increase the number of records to ensure comprehensive coverage

Switch to a different generative AI service without evaluating its relevance

Accept the high cost and time because relevance is high

Reduce the number of records generated while maintaining relevance

What does a user satisfaction metric for generative AI outputs primarily measure?

The number of outputs generated

The degree to which users find the outputs useful and acceptable

The computational cost of generating the outputs

Certified Tester Testing with Generative AI

Domain 2Objective 8

Metrics for Evaluating the Results of Generative AI on Test Tasks CT-GENAI Practice Questions (Page 6)

Part of the Prompt Engineering for Effective Software Testing domain, which makes up ~30% of our current practice bank. ISTQB does not publish an official question count, but from its 60-minute exam (~25–40 total, ~8–12 in this domain), expect 1–1 from this objective — we provide 35 practice questions to prepare you well beyond it. (estimate)

35questions here

7free pages

9concepts

Questions 26–30

26foundation · easy
What does a completeness metric for generated test cases typically measure?
Select an answer first
27foundation · easy
When evaluating a generated defect report, which metric is most important to assess its usefulness for developers?
Select an answer first
28application · medium
A QA team uses a generative AI tool to create test cases for a new payment module. They want to ensure the generated suite exercises all the acceptance criteria in the user stories, including boundary values for transaction amounts. Which metric should they prioritize to evaluate the generated test cases?
Select an answer first
29expert · hard
A team uses a generative AI service to create test data for a large-scale load test. The relevance score is high (90%), but the cost per record is high, and the generation time is long. The team has a limited budget and a tight deadline. What should they prioritize?
Select an answer first
30foundation · easy
What does a user satisfaction metric for generative AI outputs primarily measure?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to CT-GENAI

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by ISTQB. “CT-GENAI” is a trademark of its owner, used for identification only.