Techniques for Evaluating and Iteratively Refining Prompts CT-GENAI Practice Questions (Page 4)

Questions 16–20

1. 16foundation · easy

   What is the risk of making a prompt too specific for a testing task?

   • AIt will increase the model's creativity
   • BIt may overfit to narrow scenarios and miss broader cases
   • CIt will always produce incorrect outputs
   • DIt will make the prompt easier to evaluate
   Reveal Solution Discussion Select an answer first
2. 17expert · hard

   A security tester is using an AI to generate adversarial inputs for a web form. The prompt is: 'Generate malicious inputs for the search field.' The output includes common XSS payloads but misses encoded variants and inputs that exploit Unicode normalization. The tester wants to improve the prompt to handle these adversarial edge cases. Which refinement is most effective?

   • ARemove the word 'malicious' and ask for 'all possible inputs'.
   • BIncrease the number of outputs from 10 to 50 in the prompt.
   • CAsk the model to 'think of everything an attacker might do'.
   • DAdd explicit instructions to include encoded payloads (e.g., URL-encoded, HTML-encoded) and Unicode normalization edge cases.
   Reveal Solution Discussion Select an answer first
4. 18expert · hard

   A tester is evaluating two different prompts for generating test cases for a REST API. Prompt A produces 20 cases, 15 of which are relevant and cover 5 of 8 required categories. Prompt B produces 30 cases, 20 of which are relevant and cover 7 of 8 required categories. The tester wants to choose the better prompt. Which evaluation metric should be the primary deciding factor?

   • ACategory coverage: Prompt B covers 7 of 8 required categories, while Prompt A covers only 5.
   • BThe ratio of relevant to irrelevant cases: Prompt A has a better ratio (3:1 vs. 2:1).
   • CTotal number of cases: Prompt B produces more cases (30 vs. 20).
   • DRelevance rate: Prompt A has 75% relevance (15/20), while Prompt B has 66.7% (20/30).
   Reveal Solution Discussion Select an answer first
5. 19application · medium

   A team is refining a prompt for generating regression test cases. The first version produces cases that are too high-level. They add a sentence specifying 'include exact input values and expected outputs.' The second version produces more detailed cases, but some are now too specific to a single configuration. What is the most appropriate next step in the iterative refinement cycle?

   • AReview the second version's output, identify cases that are overfitted, and adjust the prompt to balance detail with general applicability.
   • BRun the second version multiple times and hope for better results.
   • CRevert to the first version because it was more general.
   • DAccept the second version as final since it is more detailed.
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   What is the purpose of designing prompts to handle adversarial inputs in software testing?

   • ATo reduce the number of test cases generated
   • BTo make the prompt longer and more complex
   • CTo increase the model's creativity
   • DTo ensure the model produces reliable outputs even with unusual or malicious inputs
   Reveal Solution Discussion Select an answer first