Agentic AI and Model-To-Model Attacks COASP Practice Questions (Page 3)

Questions 11–15

1. 11application · medium

   A company is evaluating whether to deploy an agentic AI system to automate customer support. The system would use a language model to understand queries, a planning module to decide on actions, and a tool to access the customer database. Which characteristic is most indicative of an agentic AI system?

   • AThe system uses a large language model to generate responses
   • BThe system is trained on a large dataset of customer interactions
   • CThe system can autonomously decide to call the database tool based on the user query
   • DThe system provides a chat interface for users
   Reveal Solution Discussion Select an answer first
2. 12expert · hard

   A healthcare company uses an agentic AI system to manage patient records. The system uses a language model to extract data from clinical notes and a separate model to classify the data for storage. The security team has discovered a prompt injection vulnerability that could allow an attacker to alter the classification model's behavior, causing sensitive data to be stored in an unsecured location. The team must implement a mitigation that complies with data protection regulations. Which approach is most appropriate?

   • AEncrypt all data before it is processed by the classification model
   • BImplement a human-in-the-loop review for any data that is classified as 'sensitive'
   • CImplement a data loss prevention (DLP) system that monitors where data is stored and alerts if sensitive data is placed in an unsecured location
   • DUse a rule-based classifier instead of a machine learning model
   Reveal Solution Discussion Select an answer first
4. 13application · medium

   An agentic AI system uses a model to summarize news articles and another model to classify the sentiment of the summary. An attacker crafts a news article that, when summarized, contains a hidden instruction that causes the sentiment model to output 'positive' for all articles. Which type of attack is this?

   • AData poisoning
   • BDirect prompt injection
   • CIndirect prompt injection
   • DModel extraction
   Reveal Solution Discussion Select an answer first
5. 14expert · hard

   A security team is hardening an agentic AI system that uses a planning model, a memory store, and a code execution tool. The team has identified that indirect prompt injection via the memory store is a major risk. They are considering two mitigations: (1) sanitizing all content before it is stored in memory, and (2) validating the memory content before it is used by the planning model. Which approach is more effective?

   • AValidating memory content before use is more effective because it catches malicious content that may have been stored
   • BBoth are equally effective and should be used together
   • CSanitizing content before storage is more effective because it prevents malicious content from entering the system
   • DNeither is effective; the only solution is to disable the memory store
   Reveal Solution Discussion Select an answer first
6. 15application · medium

   A security team is testing an agentic AI system that uses a language model to control a robotic arm. An attacker sends a specially crafted image to the vision model, which then generates a control command that causes the arm to move dangerously. Which type of attack is this?

   • ADirect prompt injection into the language model
   • BData poisoning of the vision model's training data
   • CIndirect prompt injection via the robotic arm's sensor data
   • DAdversarial manipulation of the vision model's output
   Reveal Solution Discussion Select an answer first