Fine-Tuning LLMs for Test Tasks CT-GENAI Practice Questions (Page 6)

Questions 26–28

1. 26foundation · easy

   Which scenario is a strong indicator that fine-tuning is needed instead of prompt engineering?

   • AThe team needs a quick prototype to explore whether an LLM can generate test ideas
   • BThe team has limited access to labeled test data
   • CThe base model, even with detailed prompts, produces test cases that do not follow the project's required format
   • DThe task is a one-time analysis of a single bug report
   Reveal Solution Discussion Select an answer first
2. 27application · medium

   A team is fine-tuning an LLM to classify bug reports by severity. They have prepared a dataset and selected a base model. What should they do to ensure the fine-tuned model performs well on unseen bug reports?

   • AEvaluate the model on the training set to check accuracy
   • BSplit the dataset into training, validation, and test sets, then evaluate on the test set
   • CTrain on the entire dataset to maximize learning
   • DUse only the training set and rely on the model's generalization
   Reveal Solution Discussion Select an answer first
3. 28foundation · easy

   Which metric is most appropriate for evaluating a fine-tuned LLM that generates test cases?

   • AThe percentage of generated test cases that are syntactically valid and executable
   • BModel latency in milliseconds
   • CThe size of the training dataset in gigabytes
   • DThe number of parameters in the model
   Reveal Solution Discussion Select an answer first