What is a method to evaluate the effectiveness of a generative AI strategy?

Asking testers to guess if AI is working

Conducting periodic reviews comparing testing outcomes before and after AI adoption

Counting the number of AI-generated test cases without checking their quality

Relying on vendor claims about AI benefits

A company is using a generative AI tool to create test data for a new application. The tool sometimes generates data that includes biased or inappropriate content. The team is concerned about the ethical implications. What should the team do to address this risk?

Implement a human review process for all AI-generated test data before it is used.

Adjust the AI model's parameters to eliminate all bias.

Stop using the AI tool and return to manual test data creation.

Ignore the biased content because it is only test data and does not affect production.

Which metric is commonly used to evaluate the effectiveness of a generative AI strategy in testing?

Number of testers who left the organization

Percentage of test cases generated by AI that pass on the first run

Time saved in test creation and execution

A large enterprise has a mature testing organization with well-defined processes and a strong data governance framework. They are considering adopting generative AI for test data generation. However, their testing team is skeptical and concerned about job security. The budget is available, but the team lacks AI skills. What is the most critical readiness factor to address first?

The availability of high-quality training data.

The maturity of the existing testing processes.

The selection of the right AI tool.

The team's resistance to change and lack of AI skills.

A company has implemented a generative AI tool to assist with test case generation. After six months, the team wants to evaluate whether the tool is meeting the strategic goal of improving test coverage. Which evaluation method would be most appropriate?

Check if the AI tool is being used by all team members.

Compare the code coverage achieved with AI-generated tests to the coverage achieved with manual tests.

Measure the number of test cases generated by the AI tool.

Survey the testing team to ask if they feel the tool is useful.

Certified Tester Testing with Generative AI

Domain 5Objective 2

Key Aspects of a Generative AI Strategy in Software Testing CT-GENAI Practice Questions (Page 4)

Part of the Deploying and Integrating Generative AI in Test organizations domain, which makes up ~23% of our current practice bank. ISTQB does not publish an official question count, but from its 60-minute exam (~25–40 total, ~6–9 in this domain), expect 1–1 from this objective — we provide 29 practice questions to prepare you well beyond it. (estimate)

29questions here

6free pages

8concepts

Questions 16–20

16foundation · easy
What is a method to evaluate the effectiveness of a generative AI strategy?
Select an answer first
17application · medium
A company is using a generative AI tool to create test data for a new application. The tool sometimes generates data that includes biased or inappropriate content. The team is concerned about the ethical implications. What should the team do to address this risk?
Select an answer first
18foundation · easy
Which metric is commonly used to evaluate the effectiveness of a generative AI strategy in testing?
Select an answer first
19expert · hard
A large enterprise has a mature testing organization with well-defined processes and a strong data governance framework. They are considering adopting generative AI for test data generation. However, their testing team is skeptical and concerned about job security. The budget is available, but the team lacks AI skills. What is the most critical readiness factor to address first?
Select an answer first
20application · medium
A company has implemented a generative AI tool to assist with test case generation. After six months, the team wants to evaluate whether the tool is meeting the strategic goal of improving test coverage. Which evaluation method would be most appropriate?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to CT-GENAI

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by ISTQB. “CT-GENAI” is a trademark of its owner, used for identification only.