A team notices that the number of failed tests in their CI pipeline has been slowly increasing over the last two weeks. They want to determine whether this is a real degradation or just random variation. Which statistical approach should they use?

Look at the latest test run only and decide based on that single result.

Calculate the moving average of test failures and compare it to the historical baseline to identify a trend.

Count the total number of tests executed over the two weeks and compare to the total from the previous month.

Ask the testers for their opinion on whether the failures are increasing.

A DevOps team wants to provide real-time visibility into the quality of each build as it progresses through the CI/CD pipeline. They currently publish a static HTML report at the end of the pipeline, but stakeholders want to see pass/fail status and trend information as soon as each stage completes. Which approach best meets this requirement?

Generate a PDF report at the end of the pipeline and upload it to a document management system.

Store test results in a shared folder and ask stakeholders to check the folder manually after each stage.

Keep the static HTML report but email it to stakeholders after the final stage completes.

Configure the pipeline to publish a machine-readable test report (e.g., JUnit XML) to a central dashboard that aggregates results from each stage in real time.

A team wants to use statistical analysis of test results to decide whether a new build is ready for release. They have historical data on pass rates and defect density. Which metric would best help them make a data-driven release decision?

Look at the number of tests executed and release if it is above a certain number.

Check if the current build has any failed tests and release if there are none.

Use the opinion of the most senior tester on the team to decide.

Compare the current build's pass rate and defect density against the historical baseline and use statistical significance testing to determine if the difference is meaningful.

A team is using statistical analysis to monitor the quality of their product. They have a large number of test results and want to identify which tests are most likely to fail in the future, so they can prioritize maintenance efforts. Which statistical technique is most appropriate?

Ask the testers which tests they think are most likely to fail.

Look at the most recently failed tests and assume they will fail again.

Count the total number of failures for each test over the last year and prioritize the ones with the most failures.

Use historical failure data to calculate the probability of failure for each test and rank them accordingly.

Why are statistical methods applied to test results in a DevOps context?

To replace the need for test automation

To identify trends and make data-driven decisions on quality

To automatically fix all failing tests

To make test results look more impressive to management

Certified Tester Quality in DevOps

Domain 3Objective 2

Automation in CI/CD CT-QDO Practice Questions (Page 1)

Part of the Domain 3: Automated and Manual Tasks in DevOps domain, which makes up ~21% of our current practice bank. ISTQB does not publish an official question count, but from its 60-minute exam (~25–40 total, ~5–8 in this domain), expect 1–2 from this objective — we provide 25 practice questions to prepare you well beyond it. (estimate)

25questions here

5free pages

4concepts

Questions 1–5

1application · medium
A team notices that the number of failed tests in their CI pipeline has been slowly increasing over the last two weeks. They want to determine whether this is a real degradation or just random variation. Which statistical approach should they use?
Select an answer first
2application · medium
A DevOps team wants to provide real-time visibility into the quality of each build as it progresses through the CI/CD pipeline. They currently publish a static HTML report at the end of the pipeline, but stakeholders want to see pass/fail status and trend information as soon as each stage completes. Which approach best meets this requirement?
Select an answer first
3application · medium
A team wants to use statistical analysis of test results to decide whether a new build is ready for release. They have historical data on pass rates and defect density. Which metric would best help them make a data-driven release decision?
Select an answer first
4expert · hard
A team is using statistical analysis to monitor the quality of their product. They have a large number of test results and want to identify which tests are most likely to fail in the future, so they can prioritize maintenance efforts. Which statistical technique is most appropriate?
Select an answer first
5foundation · easy
Why are statistical methods applied to test results in a DevOps context?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to CT-QDO

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by ISTQB. “CT-QDO” is a trademark of its owner, used for identification only.