A company is evaluating the total cost of ownership for an LLM versus an SLM for test case generation. The LLM requires a $50,000 GPU server and $0.02 per API call, while the SLM can run on existing hardware and costs $0.001 per API call. The company expects to generate 100,000 test cases per year. What is the most important factor in the cost-benefit analysis?

The total cost of ownership over the expected lifespan, including hardware and API costs

The speed of the model in generating test cases

The accuracy of the model on test case generation

The ease of integration with existing test frameworks

In test automation, what is the primary trade-off when choosing a large language model over a small language model?

Lower latency versus lower accuracy

Lower cost versus lower privacy

Better integration versus worse performance

Higher capability versus higher resource consumption

Why is continuous evaluation of a language model's performance important in a testing context?

To detect performance degradation over time

To reduce the model's training time

To simplify the integration process

To increase the model's token limit

When deploying a language model for testing, which deployment factor is most directly related to data privacy?

On-premises vs cloud deployment

Certified Tester Testing with Generative AI

Domain 5Objective 3

Selecting LLMs/SLMs for Software Test Tasks CT-GENAI Practice Questions (Page 3)

Part of the Deploying and Integrating Generative AI in Test organizations domain, which makes up ~23% of our current practice bank. ISTQB does not publish an official question count, but from its 60-minute exam (~25–40 total, ~6–9 in this domain), expect 1–1 from this objective — we provide 30 practice questions to prepare you well beyond it. (estimate)

30questions here

6free pages

10concepts

Questions 11–15

11application · medium
A company is evaluating the total cost of ownership for an LLM versus an SLM for test case generation. The LLM requires a $50,000 GPU server and $0.02 per API call, while the SLM can run on existing hardware and costs $0.001 per API call. The company expects to generate 100,000 test cases per year. What is the most important factor in the cost-benefit analysis?
Select an answer first
12foundation · easy
In test automation, what is the primary trade-off when choosing a large language model over a small language model?
Select an answer first
13foundation · easy
Why is continuous evaluation of a language model's performance important in a testing context?
Select an answer first
14foundation · easy
When deploying a language model for testing, which deployment factor is most directly related to data privacy?
Select an answer first
15foundation · easy
When benchmarking language models for test case generation, which metric is most appropriate to measure the usefulness of generated test cases?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to CT-GENAI

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by ISTQB. “CT-GENAI” is a trademark of its owner, used for identification only.

Selecting LLMs/SLMs for Software Test Tasks CT-GENAI Practice Questions (Page 3)

Questions 11–15

In test automation, what is the primary trade-off when choosing a large language model over a small language model?

Why is continuous evaluation of a language model's performance important in a testing context?

When deploying a language model for testing, which deployment factor is most directly related to data privacy?

When benchmarking language models for test case generation, which metric is most appropriate to measure the usefulness of generated test cases?

Keep studying — Deploying and Integrating Generative AI in Test organizations