Techniques for Evaluating and Iteratively Refining Prompts CT-GENAI Practice Questions (Page 6)

Questions 26–30

1. 26foundation · easy

   Which sequence best describes a systematic iterative refinement cycle for improving a prompt's output?

   • AAnalyze errors, modify prompt, evaluate output, stop
   • BModify prompt, evaluate output, analyze errors, repeat
   • CDeploy prompt, collect feedback, modify, evaluate
   • DEvaluate output, modify prompt, deploy, stop
   Reveal Solution Discussion Select an answer first
2. 27application · medium

   A QA team has been refining a prompt for generating test cases for a microservices architecture. They have made five versions, each with different performance. They need to present the evolution to stakeholders and justify the final prompt. What documentation approach best supports this?

   • AShow the raw chat logs of all the prompt iterations.
   • BDescribe the evolution verbally without any written record.
   • CCreate a table showing each version, the changes made, the rationale, and the evaluation metrics (e.g., coverage, relevance) for each.
   • DPresent only the final prompt and explain why it is the best.
   Reveal Solution Discussion Select an answer first
4. 28foundation · easy

   What should be included in documentation of prompt evolution to support reproducibility?

   • AOnly the final prompt version
   • BThe exact prompt text, the changes made, and the evaluation results
   • CThe number of tokens in each version
   • DThe model's internal parameters
   Reveal Solution Discussion Select an answer first
5. 29application · medium

   A team uses an AI to generate user stories for a new feature. After reviewing the first batch, they find that the stories lack acceptance criteria. They add a line to the prompt: 'Include acceptance criteria for each story.' The second batch now includes acceptance criteria, but they are too generic. The team wants to continue improving the prompt using feedback. What should they do next?

   • ARevert to the first prompt because the second one is too generic.
   • BAccept the second batch as final since it now includes acceptance criteria.
   • CRun the second prompt again and hope for more specific criteria.
   • DAnalyze the generic acceptance criteria, identify what makes them generic, and refine the prompt to require specific, testable criteria.
   Reveal Solution Discussion Select an answer first
6. 30expert · hard

   An AI prompt for generating test cases for a search function consistently produces cases that assume the search is case-sensitive, even though the actual implementation is case-insensitive. The tester has already tried adding 'remember the search is case-insensitive' to the prompt, but the output is unchanged. What is the most likely root cause and the most effective next refinement?

   • AThe model is anchored by common assumptions; provide explicit examples of case-insensitive scenarios (e.g., 'Search for 'Apple' should match 'apple'') in the prompt.
   • BThe model is ignoring the instruction due to prompt length; shorten the prompt to the bare minimum.
   • CThe model is not capable of understanding case sensitivity; switch to a different AI tool.
   • DThe model is biased by the word 'search'; replace it with 'lookup' in the prompt.
   Reveal Solution Discussion Select an answer first