A team uses an LLM API for test generation. They have a monthly budget and want to ensure they do not exceed it. They also want to avoid any interruption to their CI/CD pipeline. What is the best strategy?

Implement a caching mechanism to reuse previously generated test cases.

Use a combination of budget alerts, caching, and model optimization to reduce costs while maintaining pipeline reliability.

Set up a budget alert and a hard limit that stops API calls when the budget is exceeded.

Use a cheaper model and reduce the number of test cases.

Why is it important to assess LLM outputs for bias when using them to generate test cases?

To reduce the number of test cases needed

To make the LLM generate more complex test cases

To ensure the test cases cover diverse scenarios and do not discriminate against certain user groups

To increase the speed of test execution

Which monitoring metric is most directly used to detect model drift in an LLM used for test generation?

Distribution of output text similarity compared to a baseline

Number of API requests per second

Latency of individual inference requests

CPU utilization of the inference server

A team is using an LLM to generate test data for a hiring platform. They are concerned about the model producing biased test cases that could lead to discriminatory outcomes in the hiring process. They need to ensure fairness while maintaining the efficiency of the test generation. What is the best approach?

Implement a bias detection and mitigation pipeline that evaluates the generated test cases for fairness and automatically adjusts the prompts or filters biased cases.

Add a disclaimer to the test reports that the data may be biased.

Use a more powerful LLM that is known to be less biased.

Use a human reviewer to manually check every generated test case for bias.

A team wants to integrate an LLM-based test data generator into their CI/CD pipeline. They need to ensure that the model version used in production is the same as the one tested in staging. What is the best practice?

Deploy the model to production and staging from the same versioned artifact registry entry.

Retrain the model in each environment to ensure freshness.

Manually copy the model files to each environment.

Use the latest model version in production and the previous version in staging.

Certified Tester Testing with Generative AI

Domain 4Objective 5

LLMOps When Deploying and Managing LLMs for Software Testing CT-GENAI Practice Questions (Page 2)

Part of the LLM-Powered Test Infrastructure for Software Testing domain, which makes up ~18% of our current practice bank. ISTQB does not publish an official question count, but from its 60-minute exam (~25–40 total, ~5–7 in this domain), expect 1–1 from this objective — we provide 36 practice questions to prepare you well beyond it. (estimate)

36questions here

8free pages

11concepts

11 Concepts in This Objective

LLMOps fundamentals
LLM lifecycle management
Model versioning and registry
Deployment strategies for LLMs
Scaling and resource management
Monitoring and observability
Evaluation and quality assurance
Security and compliance
Cost optimization
Integration with CI/CD
Governance and ethics

Questions 6–10

6expert · hard
A team uses an LLM API for test generation. They have a monthly budget and want to ensure they do not exceed it. They also want to avoid any interruption to their CI/CD pipeline. What is the best strategy?
Select an answer first
7foundation · easy
Why is it important to assess LLM outputs for bias when using them to generate test cases?
Select an answer first
8foundation · easy
Which monitoring metric is most directly used to detect model drift in an LLM used for test generation?
Select an answer first
9expert · hard
A team is using an LLM to generate test data for a hiring platform. They are concerned about the model producing biased test cases that could lead to discriminatory outcomes in the hiring process. They need to ensure fairness while maintaining the efficiency of the test generation. What is the best approach?
Select an answer first
10application · medium
A team wants to integrate an LLM-based test data generator into their CI/CD pipeline. They need to ensure that the model version used in production is the same as the one tested in staging. What is the best practice?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to CT-GENAI

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by ISTQB. “CT-GENAI” is a trademark of its owner, used for identification only.