Techniques for Evaluating and Iteratively Refining Prompts CT-GENAI Practice Questions (Page 1)

Questions 1–5

1. 1foundation · easy

   When evaluating the outputs of a prompt that generates test data, which qualitative criterion is most relevant to assess whether the data covers a wide range of input conditions?

   • AConsistency
   • BDiversity
   • CFluency
   • DLatency
   Reveal Solution Discussion Select an answer first
2. 2expert · hard

   A tester is using an AI assistant to generate negative test cases for an API endpoint. The prompt is: 'Generate negative test cases for the /login endpoint.' The output repeatedly produces only cases like 'invalid password' and 'empty username,' missing cases like 'SQL injection in username field' and 'extremely long input.' The tester wants to diagnose the root cause of this narrow output. Which is the most likely underlying cause, and what is the most effective refinement?

   • AThe model is overfitting to the word 'negative'; refine the prompt by removing the word and asking for 'test cases' instead.
   • BThe prompt lacks sufficient context about what 'negative' means; refine it to explicitly list categories such as injection, boundary, and malformed data.
   • CThe model is biased toward common cases; refine the prompt by asking for 'unusual' cases only.
   • DThe model is failing because the endpoint is not described; refine the prompt to include the full API specification.
   Reveal Solution Discussion Select an answer first
4. 3application · medium

   A team uses an AI tool to generate test steps for a mobile app. After each sprint, they manually review the generated steps and find that some are outdated due to UI changes. They want to incorporate this feedback into the prompt for the next sprint. Which approach best implements a feedback loop?

   • AKeep the prompt unchanged and manually fix the outdated steps after generation.
   • BAsk the AI to review its own previous output and correct it.
   • CDelete the old prompt and write a completely new one from scratch each sprint.
   • DUpdate the prompt to include the latest UI specification and add a note that previously generated steps may be outdated.
   Reveal Solution Discussion Select an answer first
5. 4foundation · easy

   Which prompt design technique is most effective for making a test case generation prompt robust to unexpected input formats?

   • AIncluding explicit instructions on how to handle invalid or unexpected inputs
   • BAvoiding any mention of edge cases
   • CIncreasing the model's temperature
   • DUsing vague language to allow flexibility
   Reveal Solution Discussion Select an answer first
6. 5expert · hard

   A security tester is using an AI to generate fuzz inputs for a file parser. The prompt is: 'Generate fuzz inputs for the file parser.' The output contains mostly random binary data but misses structured malformed files (e.g., truncated headers, incorrect checksums). The tester suspects the prompt is too vague. Which root cause analysis and refinement is most accurate?

   • AThe model is biased toward random data; refine the prompt to ask for 'structured' inputs.
   • BThe prompt lacks context about the file format; refine it to specify the file structure and ask for malformed variations of that structure.
   • CThe prompt is too long; shorten it to 'Generate fuzz inputs.'
   • DThe model is not capable of generating structured fuzz inputs; switch to a specialized fuzzing tool.
   Reveal Solution Discussion Select an answer first