A team uses generative AI to generate test data for a database migration test. They run the same prompt multiple times and find that the generated data varies significantly, leading to inconsistent test results. Which metric should they use to evaluate this issue?

Relevance of the data to the migration test

A team uses generative AI to synthesize test data for a login form. They need the data to include valid and invalid email formats, boundary-length passwords, and Unicode characters. Which metric best evaluates whether the generated data is suitable for this specific testing purpose?

Time taken to generate the dataset

Relevance of the data to the input validation requirements

Consistency of data format across runs

A team uses a generative AI service to create test data for a large-scale performance test. They are concerned about the cost and time required to generate the data, as the service charges per API call. Which metric should they track to evaluate the efficiency of the generation process?

Number of data records generated

Average cost per generated data record

Relevance of data to test scenarios

What does an accuracy metric for a generated test case primarily evaluate?

How consistent the test case is with previously generated test cases

Whether the test case's expected results align with the specification

How well the test case covers all possible input combinations

How quickly the test case was generated

Certified Tester Testing with Generative AI

Domain 2Objective 8

Metrics for Evaluating the Results of Generative AI on Test Tasks CT-GENAI Practice Questions (Page 1)

Part of the Prompt Engineering for Effective Software Testing domain, which makes up ~30% of our current practice bank. ISTQB does not publish an official question count, but from its 60-minute exam (~25–40 total, ~8–12 in this domain), expect 1–1 from this objective — we provide 35 practice questions to prepare you well beyond it. (estimate)

35questions here

7free pages

9concepts

Questions 1–5

1application · medium
A team uses generative AI to generate test data for a database migration test. They run the same prompt multiple times and find that the generated data varies significantly, leading to inconsistent test results. Which metric should they use to evaluate this issue?
Select an answer first
2foundation · easy
Which metric is used to assess the extent to which generated test cases exercise different branches of the code?
Select an answer first
3application · medium
A team uses generative AI to synthesize test data for a login form. They need the data to include valid and invalid email formats, boundary-length passwords, and Unicode characters. Which metric best evaluates whether the generated data is suitable for this specific testing purpose?
Select an answer first
4application · medium
A team uses a generative AI service to create test data for a large-scale performance test. They are concerned about the cost and time required to generate the data, as the service charges per API call. Which metric should they track to evaluate the efficiency of the generation process?
Select an answer first
5foundation · easy
What does an accuracy metric for a generated test case primarily evaluate?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to CT-GENAI

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by ISTQB. “CT-GENAI” is a trademark of its owner, used for identification only.