Scoping an AI Red Team Exercise and Threat Modeling COASP Practice Questions (Page 8)

Questions 36–37

1. 36application · medium

   A red team is planning an exercise for an AI-powered fraud detection system used by a bank. The team has identified three threats: (1) an attacker evades the model with crafted transactions, (2) an insider steals the model weights, and (3) a denial-of-service attack on the scoring API. The bank's primary concern is financial loss from undetected fraud. Which threat should the red team prioritize?

   • AModel evasion, because it directly undermines the system's core purpose of detecting fraud.
   • BDenial of service on the API, because it makes the system unavailable.
   • CModel weight theft, because it compromises the intellectual property of the model.
   • DAll three threats should be given equal priority.
   Reveal Solution Discussion Select an answer first
2. 37application · medium

   A red team is using MITRE ATLAS to threat model an AI system that uses a large language model for code generation. The team identifies a threat where an attacker crafts a malicious prompt that causes the model to generate vulnerable code, which is then used in production. Which MITRE ATLAS technique best describes this attack?

   • AML Model Inference API Access, because the attacker is accessing the model's API.
   • BML Supply Chain Compromise, because the model is compromised during development.
   • CData Poisoning, because the attacker is corrupting the training data.
   • DPrompt Injection, because the attacker is manipulating the model's input to control its output.
   Reveal Solution Discussion Select an answer first