Fine-Tuning LLMs for Test Tasks CT-GENAI Practice Questions (Page 3)

Questions 11–15

1. 11foundation · easy

   Which scenario best illustrates the benefit of fine-tuning an LLM for defect analysis?

   • AThe model consistently classifies bug reports into the project's specific severity categories
   • BThe model can translate bug reports into multiple languages
   • CThe model can answer general questions about software testing
   • DThe model generates test data faster than a human tester
   Reveal Solution Discussion Select an answer first
2. 12foundation · easy

   What does 'coverage' measure when evaluating a fine-tuned LLM for test generation?

   • AThe percentage of test cases that pass when executed
   • BThe extent to which the generated tests exercise different parts of the code or requirements
   • CThe amount of training data used for fine-tuning
   • DThe number of test cases generated per minute
   Reveal Solution Discussion Select an answer first
4. 13expert · hard

   A team is fine-tuning an LLM to generate test cases for a mobile app. They have a dataset of 5,000 test cases. During training, they notice that the validation loss decreases initially but then increases after a few epochs. What should they do?

   • AStop training early at the point where validation loss is lowest
   • BReduce the size of the training set
   • CIncrease the learning rate to escape the local minimum
   • DContinue training for more epochs to see if validation loss decreases again
   Reveal Solution Discussion Select an answer first
5. 14application · medium

   A team has fine-tuned an LLM to generate test cases for a web application. They want to evaluate the model's performance. Which combination of metrics would provide the most comprehensive assessment?

   • AExecution time and memory usage
   • BPrecision and recall only
   • CBLEU score and perplexity
   • DAccuracy, relevance, and coverage
   Reveal Solution Discussion Select an answer first
6. 15application · medium

   A team is fine-tuning an LLM to generate test cases for a healthcare application. The training dataset contains mostly test cases for the login and registration features, with very few for other features. What is the most likely risk, and how should they mitigate it?

   • AData leakage; remove the test set from the training data
   • BData bias; balance the dataset across features
   • CUnderfitting; increase the number of training epochs
   • DOverfitting; use a smaller learning rate
   Reveal Solution Discussion Select an answer first