What is the first step in the fine-tuning process for an LLM?

Deploying the model to production

Setting the learning rate to a high value

Evaluating the model on the test set

Selecting a base model that is appropriate for the task

A team is curating a dataset for fine-tuning an LLM to generate test cases for a mobile app. They have collected test cases from various sources, including some that are outdated or no longer relevant to the current app version. What should they do to ensure the dataset is suitable for fine-tuning?

Convert all test cases to a single format

Label each test case with its source

Remove outdated and irrelevant test cases

Keep all test cases to maximize dataset size

When is prompt engineering generally more suitable than fine-tuning for a test-related task?

When the task is a one-off or infrequent need and the model can perform adequately with well-crafted instructions

When the task requires the model to learn a new, domain-specific output format that is not in the model's training data

When the task involves processing a large volume of test cases with high accuracy requirements

When the model must be deployed in an offline environment with no API access

A team is fine-tuning an LLM to generate test cases for a safety-critical system. They have a limited dataset of high-quality test cases. They must ensure the model does not generate unsafe or invalid test cases. What is the most important consideration during the fine-tuning process?

Maximize the number of training epochs to improve accuracy

Use a conservative evaluation strategy with human review of generated test cases

Use a large learning rate to speed up training

Rely solely on automated metrics like BLEU score

A team is preparing a dataset for fine-tuning an LLM to generate test cases for a financial application. They have collected test cases from multiple sources, but some test cases contain sensitive customer data. What should they do to ensure the dataset is safe and compliant?

Encrypt the dataset and use it as-is

Anonymize or remove sensitive data from the test cases

Keep the sensitive data to make the test cases more realistic

Certified Tester Testing with Generative AI

Domain 4Objective 4

Fine-Tuning LLMs for Test Tasks CT-GENAI Practice Questions (Page 2)

Part of the LLM-Powered Test Infrastructure for Software Testing domain, which makes up ~18% of our current practice bank. ISTQB does not publish an official question count, but from its 60-minute exam (~25–40 total, ~5–7 in this domain), expect 1–1 from this objective — we provide 28 practice questions to prepare you well beyond it. (estimate)

28questions here

6free pages

6concepts

Questions 6–10

6foundation · easy
What is the first step in the fine-tuning process for an LLM?
Select an answer first
7application · medium
A team is curating a dataset for fine-tuning an LLM to generate test cases for a mobile app. They have collected test cases from various sources, including some that are outdated or no longer relevant to the current app version. What should they do to ensure the dataset is suitable for fine-tuning?
Select an answer first
8foundation · easy
When is prompt engineering generally more suitable than fine-tuning for a test-related task?
Select an answer first
9expert · hard
A team is fine-tuning an LLM to generate test cases for a safety-critical system. They have a limited dataset of high-quality test cases. They must ensure the model does not generate unsafe or invalid test cases. What is the most important consideration during the fine-tuning process?
Select an answer first
10application · medium
A team is preparing a dataset for fine-tuning an LLM to generate test cases for a financial application. They have collected test cases from multiple sources, but some test cases contain sensitive customer data. What should they do to ensure the dataset is safe and compliant?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to CT-GENAI

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by ISTQB. “CT-GENAI” is a trademark of its owner, used for identification only.