A team is planning an A/B test to measure the impact of a new AI-based recommendation engine on click-through rate (CTR). They estimate the baseline CTR is 2% and want to detect a 10% relative increase (to 2.2%). They set power to 80% and significance level to 5%. What is the most important next step?

Calculate the required sample size using a power analysis tool.

Launch the test immediately to collect as much data as possible.

Use a larger effect size to make the test easier to pass.

Increase the significance level to 0.10 to make the test easier.

Certified Tester AI Testing

Domain 9Objective 5

A/B Testing CT-AI Practice Questions (Page 6)

Part of the Domain 9: Methods and Techniques for the Testing of AI-Based Systems domain, which makes up ~13% of our current practice bank. ISTQB does not publish an official question count, but from its 60-minute exam (~25–40 total, ~3–5 in this domain), expect 1–1 from this objective — we provide 26 practice questions to prepare you well beyond it. (estimate)

26questions here

6free pages

9concepts

Questions 26–26

26application · medium
A team is planning an A/B test to measure the impact of a new AI-based recommendation engine on click-through rate (CTR). They estimate the baseline CTR is 2% and want to detect a 10% relative increase (to 2.2%). They set power to 80% and significance level to 5%. What is the most important next step?
Select an answer first

Finished these 1 questions?

Review the revealed explanations, or continue through the curriculum.

No more pagesBack to CT-AI

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by ISTQB. “CT-AI” is a trademark of its owner, used for identification only.

A/B Testing CT-AI Practice Questions (Page 6)

Questions 26–26

Keep studying — Domain 9: Methods and Techniques for the Testing of AI-Based Systems