Which format is commonly used to structure a fine-tuning dataset for LLMs?

JSONL with each line containing an input-output pair

CSV with a single column for the entire test suite

Plain text with no delimiters between examples

XML with nested tags for each test case

A QA team uses a general-purpose LLM to generate test cases for a legacy banking application. The LLM frequently produces test cases that are syntactically valid but miss domain-specific business rules, such as interest calculation formulas and regulatory limits. The team has a large, well-labeled repository of historical test cases that cover these rules. They want to improve the LLM's output without changing the model's architecture. What should they do?

Apply a different sampling temperature during generation

Fine-tune the LLM on the historical test case repository

Switch to a larger general-purpose LLM

Use a longer system prompt with more detailed instructions

A team needs to generate test cases for a new feature that is similar to an existing feature. They have a large dataset of test cases for the existing feature. They want to minimize the cost and time of customization. What is the most appropriate approach?

Use a rule-based test generator

Use prompt engineering with a few examples

Fine-tune the LLM on the existing feature's test cases

Use a general-purpose LLM without customization

Which practice is a best practice for fine-tuning an LLM for software testing?

Setting the learning rate as high as possible to speed up training

Avoiding any validation step to save time

Training on only the most recent bug reports to keep the model up to date

Using a training dataset that reflects the diversity of real-world testing scenarios

A startup has a small QA team and needs to generate test cases for a new mobile app. They have only a few dozen example test cases and the app's requirements change weekly. They want to quickly adapt the LLM's output to new features without retraining. Which approach is most suitable?

Use prompt engineering with detailed instructions and few-shot examples

Use a larger LLM without any customization

Build a custom rule-based test generator

Fine-tune the LLM on the few dozen examples

Certified Tester Testing with Generative AI

Domain 4Objective 4

Fine-Tuning LLMs for Test Tasks CT-GENAI Practice Questions (Page 5)

Part of the LLM-Powered Test Infrastructure for Software Testing domain, which makes up ~18% of our current practice bank. ISTQB does not publish an official question count, but from its 60-minute exam (~25–40 total, ~5–7 in this domain), expect 1–1 from this objective — we provide 28 practice questions to prepare you well beyond it. (estimate)

28questions here

6free pages

6concepts

Questions 21–25

21foundation · easy
Which format is commonly used to structure a fine-tuning dataset for LLMs?
Select an answer first
22application · medium
A QA team uses a general-purpose LLM to generate test cases for a legacy banking application. The LLM frequently produces test cases that are syntactically valid but miss domain-specific business rules, such as interest calculation formulas and regulatory limits. The team has a large, well-labeled repository of historical test cases that cover these rules. They want to improve the LLM's output without changing the model's architecture. What should they do?
Select an answer first
23application · medium
A team needs to generate test cases for a new feature that is similar to an existing feature. They have a large dataset of test cases for the existing feature. They want to minimize the cost and time of customization. What is the most appropriate approach?
Select an answer first
24foundation · easy
Which practice is a best practice for fine-tuning an LLM for software testing?
Select an answer first
25application · medium
A startup has a small QA team and needs to generate test cases for a new mobile app. They have only a few dozen example test cases and the app's requirements change weekly. They want to quickly adapt the LLM's output to new features without retraining. Which approach is most suitable?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to CT-GENAI

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by ISTQB. “CT-GENAI” is a trademark of its owner, used for identification only.