LLM Jailbreaking COASP Practice Questions (Page 2)

Questions 6–10

1. 6expert · hard

   A company's LLM-powered customer support chatbot is being jailbroken by users who use persona shifting to get unauthorized discounts. The company wants to mitigate this without making the chatbot less helpful. Which mitigation is most effective?

   • AAllow users to customize the chatbot's persona.
   • BEnforce a strict system prompt that prevents the chatbot from adopting user-defined personas and add a policy-enforcement layer that checks for unauthorized discount requests.
   • CUse a larger model with better instruction-following capabilities.
   • DDisable the chatbot's ability to provide discounts.
   Reveal Solution Discussion Select an answer first
2. 7application · medium

   A red team is evaluating an LLM's robustness. They use a tool that automatically generates adversarial suffixes and appends them to prompts. The tool successfully causes the model to output prohibited content. What is the most likely reason for the success, and what mitigation should be recommended?

   • AThe system prompt is too weak; add more rules to the system prompt.
   • BThe training data is insufficient; retrain the model on more safe examples.
   • CThe model is not large enough; upgrade to a larger model.
   • DThe adversarial suffix exploits token-level vulnerabilities; implement a filter that detects and blocks known adversarial patterns.
   Reveal Solution Discussion Select an answer first
4. 8application · medium

   A company is deploying an LLM that processes user-uploaded documents. They are concerned about indirect prompt injection attacks. Which mitigation strategy is most effective?

   • AAllow users to customize the system prompt.
   • BDisable logging to avoid storing sensitive data.
   • CUse a larger model with more parameters.
   • DTreat all uploaded content as untrusted data and sanitize it before it reaches the LLM.
   Reveal Solution Discussion Select an answer first
5. 9application · medium

   A red-team analyst is testing an LLM's robustness. The analyst uses an automated tool that generates a sequence of tokens that, when appended to a prompt, causes the model to output harmful content. The tool optimizes the token sequence to maximize the probability of a jailbreak. Which technique is the analyst using?

   • APayload splitting
   • BDirect prompt manipulation
   • CAdversarial suffix generation
   • DRole-playing
   Reveal Solution Discussion Select an answer first
6. 10application · medium

   A security researcher is testing an LLM's resistance to jailbreaks. The researcher appends a seemingly random string of characters to a benign prompt, and the model suddenly outputs prohibited content. What is the most likely explanation, and what mitigation should be recommended?

   • AAdversarial suffix; implement input filtering that detects and blocks known adversarial token patterns.
   • BIndirect prompt injection; sanitize external content.
   • CDirect prompt manipulation; block the word 'ignore'.
   • DRole-playing; add a system prompt that forbids role changes.
   Reveal Solution Discussion Select an answer first