Fine-Tuning LLMs for Test Tasks CT-GENAI Practice Questions (Page 1)

Questions 1–5

1. 1application · medium

   A team has fine-tuned an LLM to generate unit test cases for a Python codebase. They want to evaluate the model's output. Which metric is most appropriate to measure whether the generated test cases actually exercise the intended code paths?

   • ABLEU score
   • BPerplexity
   • CF1 score
   • DCode coverage
   Reveal Solution Discussion Select an answer first
2. 2application · medium

   A team fine-tuned an LLM on a dataset of test cases for a web application. After fine-tuning, the model performs very well on the training data but poorly on new test cases. What is the most likely cause and what should they do?

   • AUnderfitting; increase the number of training epochs
   • BModel bias; use a different base model
   • COverfitting; use a larger and more diverse dataset and apply regularization
   • DData leakage; remove the test set from the training data
   Reveal Solution Discussion Select an answer first
4. 3application · medium

   A team is preparing a dataset to fine-tune an LLM for generating API test cases. They have collected a large number of test cases from various projects, but some are duplicates, some have incorrect expected results, and others are written in different styles. What is the most important step before using this dataset for fine-tuning?

   • AUse the dataset as-is to avoid bias
   • BSplit the dataset into training and test sets
   • CIncrease the dataset size by adding more test cases
   • DDeduplicate, clean, and standardize the test cases
   Reveal Solution Discussion Select an answer first
5. 4foundation · easy

   What is a key requirement when preparing a dataset for fine-tuning an LLM for test generation?

   • AThe dataset must be at least one million examples to achieve good performance
   • BThe dataset should include a diverse set of test cases that represent the types of tests the model will generate
   • CThe dataset must contain only positive examples to avoid confusing the model
   • DThe dataset should be formatted as a single text file with no structure
   Reveal Solution Discussion Select an answer first
6. 5foundation · easy

   What is the primary purpose of fine-tuning a general-purpose LLM for software testing tasks?

   • ATo adapt the model to generate outputs that align with the style and requirements of specific testing tasks
   • BTo enable the model to run on lower-end hardware without additional configuration
   • CTo increase the model's general knowledge about software engineering
   • DTo reduce the model's inference latency during test execution
   Reveal Solution Discussion Select an answer first