Techniques for Evaluating and Iteratively Refining Prompts CT-GENAI Practice Questions (Page 3)

Questions 11–15

1. 11foundation · easy

   What is the main benefit of generating and comparing multiple prompt variants for the same testing task?

   • AIt helps identify the most effective prompt among alternatives
   • BIt eliminates the need for iterative refinement
   • CIt guarantees the model will produce correct outputs
   • DIt reduces the need for evaluation metrics
   Reveal Solution Discussion Select an answer first
2. 12application · medium

   A test engineer needs to generate test data for a date-validation function. The first prompt, 'Generate test data for the date validator,' produces mostly valid dates. The engineer wants to explore a wider range of test scenarios, including invalid, boundary, and leap-year cases. Which approach best applies the technique of prompt variation?

   • AAsk the model to generate 100 dates in one prompt without specifying categories.
   • BRun the same prompt multiple times and keep all unique outputs.
   • CCreate three separate prompts: one for valid dates, one for invalid dates, and one for boundary/leap-year cases, then combine the outputs.
   • DModify the original prompt to add 'include everything' at the end.
   Reveal Solution Discussion Select an answer first
4. 13foundation · easy

   What is the primary purpose of re-evaluating a prompt after each modification in an iterative refinement process?

   • ATo satisfy a requirement for multiple versions
   • BTo ensure the prompt is shorter
   • CTo increase the number of tokens in the prompt
   • DTo determine whether the change improved output quality
   Reveal Solution Discussion Select an answer first
5. 14application · medium

   A QA team has iterated on a prompt for generating API test cases over several weeks. A new team member joins and needs to understand why the current prompt is structured the way it is. The team wants to support reproducibility and learning. What should they do?

   • AStore the prompt in a shared document without any version history.
   • BAsk the new member to reverse-engineer the prompt from the test results.
   • CKeep only the final prompt and delete all previous versions to avoid confusion.
   • DMaintain a versioned log of the prompt, including changes made, the rationale for each change, and the performance metrics observed.
   Reveal Solution Discussion Select an answer first
6. 15application · medium

   A tester uses an AI to generate test cases for a file-upload feature. The output includes cases for valid files, but misses cases for oversized files and files with incorrect extensions. The tester wants to evaluate the prompt's effectiveness quantitatively. Which metric is most appropriate for this evaluation?

   • AThe total number of test cases generated.
   • BThe number of words in the prompt.
   • CCoverage rate: the percentage of required test categories (e.g., valid, oversized, wrong extension) that appear in the output.
   • DThe average time taken to generate the cases.
   Reveal Solution Discussion Select an answer first