Scoping an AI Red Team Exercise and Threat Modeling COASP Practice Questions (Page 3)

Questions 11–15

1. 11foundation · easy

   When prioritizing threats for an AI red team exercise, which two factors are most commonly used to assess risk?

   • ACost and speed
   • BLikelihood and impact
   • CComplexity and elegance
   • DPopularity and visibility
   Reveal Solution Discussion Select an answer first
2. 12expert · hard

   A red team is applying MITRE ATLAS to an AI system that uses a pre-trained model and a fine-tuning pipeline. The team has identified a threat where an attacker could poison the fine-tuning data. Using ATLAS, which technique would this threat be mapped to?

   • AData Exfiltration, which involves stealing sensitive data from the system.
   • BPrompt Injection, which involves manipulating the model's prompts to elicit unintended behavior.
   • CAdversarial Input, which involves crafting inputs to cause misclassification at inference time.
   • DML Model Poisoning, which involves injecting malicious data into the training or fine-tuning process to corrupt the model's behavior.
   Reveal Solution Discussion Select an answer first
4. 13foundation · easy

   In the STRIDE threat modeling framework, which category would a threat that allows an attacker to modify the training data of an AI model fall under?

   • ARepudiation
   • BDenial of Service
   • CSpoofing
   • DTampering
   Reveal Solution Discussion Select an answer first
5. 14application · medium

   A company is red teaming an AI system that uses a pre-trained language model, a fine-tuning dataset, and a vector database for retrieval-augmented generation (RAG). The system is deployed on cloud infrastructure with an API gateway. Which component is most likely to be overlooked but is a critical attack surface?

   • AThe fine-tuning dataset, because it is used to adapt the model.
   • BThe API gateway, because it is the entry point for all requests.
   • CThe pre-trained language model, because it is the core of the system.
   • DThe vector database, because it stores the knowledge base that the model retrieves from, and poisoning it could manipulate the model's responses.
   Reveal Solution Discussion Select an answer first
6. 15application · medium

   A multinational bank is planning an AI red team exercise for its loan approval model. The bank must comply with GDPR and local banking regulations that require explainability of credit decisions. The red team lead wants to ensure the exercise aligns with organizational goals. Which action best aligns the red team scope with these requirements?

   • AExclude any testing that might reveal model biases, to avoid regulatory scrutiny.
   • BInclude testing for adversarial inputs that could cause the model to make biased or unexplainable decisions, and document how such findings would impact regulatory compliance.
   • CFocus only on technical vulnerabilities in the API and infrastructure, since compliance is a legal matter, not a security concern.
   • DLimit the exercise to the model's accuracy metrics, because explainability is a separate project.
   Reveal Solution Discussion Select an answer first