A team is deploying an LLM for test automation. The workload is predictable: 10,000 test cases run every night at 2 AM. They need to minimize cost while ensuring the job completes within a 2-hour window. What is the best approach?

Provision a dedicated GPU cluster that is always on.

Schedule a batch inference job on a pre-provisioned cluster that is started just before the run and shut down after.

Use a serverless inference endpoint that scales automatically.

Run the model on a CPU-only instance to save costs.

Which practice is most effective for reducing LLM inference costs in a test environment?

Disabling monitoring to save on logging costs

Increasing the number of inference instances

Batching multiple test requests into a single inference call

Using the largest available model for all tasks

An LLM is used to generate test case descriptions from user stories. The team wants to ensure the generated descriptions are accurate and complete. Which evaluation approach is most appropriate?

Rely solely on automated metrics like BLEU score.

Compare the generated descriptions to the original user stories using a simple string match.

Use a human-in-the-loop review process where testers validate a sample of generated descriptions.

Check that the descriptions are grammatically correct using a spell checker.

A team updates the prompt template used by their LLM-based test data generator. They need to ensure that any test run can be traced back to the exact model and prompt version that produced the data. What should they implement?

Store the model weights and prompt template in a versioned artifact registry and record the artifact IDs in the test run metadata.

Log the model name and prompt text in the test report after the run.

Use a fixed model version and update the prompt in place without versioning.

Keep a copy of the prompt template in the test repository and rely on git history for versioning.

Certified Tester Testing with Generative AI

Domain 4Objective 5

LLMOps When Deploying and Managing LLMs for Software Testing CT-GENAI Practice Questions (Page 3)

Part of the LLM-Powered Test Infrastructure for Software Testing domain, which makes up ~18% of our current practice bank. ISTQB does not publish an official question count, but from its 60-minute exam (~25–40 total, ~5–7 in this domain), expect 1–1 from this objective — we provide 36 practice questions to prepare you well beyond it. (estimate)

36questions here

8free pages

11concepts

11 Concepts in This Objective

LLMOps fundamentals
LLM lifecycle management
Model versioning and registry
Deployment strategies for LLMs
Scaling and resource management
Monitoring and observability
Evaluation and quality assurance
Security and compliance
Cost optimization
Integration with CI/CD
Governance and ethics

Questions 11–15

11application · medium
A team is deploying an LLM for test automation. The workload is predictable: 10,000 test cases run every night at 2 AM. They need to minimize cost while ensuring the job completes within a 2-hour window. What is the best approach?
Select an answer first
12foundation · easy
Which stage of the LLM lifecycle involves preparing and cleaning the data used to train or fine-tune the model for testing tasks?
Select an answer first
13foundation · easy
Which practice is most effective for reducing LLM inference costs in a test environment?
Select an answer first
14application · medium
An LLM is used to generate test case descriptions from user stories. The team wants to ensure the generated descriptions are accurate and complete. Which evaluation approach is most appropriate?
Select an answer first
15application · medium
A team updates the prompt template used by their LLM-based test data generator. They need to ensure that any test run can be traced back to the exact model and prompt version that produced the data. What should they implement?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to CT-GENAI

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by ISTQB. “CT-GENAI” is a trademark of its owner, used for identification only.