Scoping an AI Red Team Exercise and Threat Modeling COASP Practice Questions (Page 7)

Questions 31–35

1. 31foundation · easy

   Which of the following is a key constraint that should be considered when scoping an AI red team exercise?

   • AThe preferred programming language of the red team
   • BThe color of the AI system's user interface
   • CThe maximum allowable impact on production systems
   • DThe number of red team members available
   Reveal Solution Discussion Select an answer first
2. 32application · medium

   A red team is threat modeling an AI-based recommendation system that uses a collaborative filtering model trained on user behavior data. The team applies STRIDE to the system. Which threat category would most directly address an attacker crafting fake user accounts to manipulate the recommendations shown to other users?

   • ADenial of service, because the fake accounts overwhelm the system.
   • BRepudiation, because the attacker can deny creating the fake accounts.
   • CTampering, because the attacker is modifying the training data through the creation of malicious accounts.
   • DSpoofing, because the attacker is impersonating legitimate users.
   Reveal Solution Discussion Select an answer first
4. 33foundation · easy

   Which of the following is an example of an AI system's attack surface that should be considered during scoping?

   • AThe organization's public website homepage
   • BThe physical locks on the server room door
   • CThe company's employee training slides
   • DThe model's inference API endpoint
   Reveal Solution Discussion Select an answer first
5. 34application · medium

   A government agency is red teaming an AI system that classifies public social media posts for threat detection. The system uses a fine-tuned language model and a separate sentiment analysis model. The agency has a strict policy that no personal data of citizens may be used in testing. Which scope is most appropriate?

   • ATest only the language model, since the sentiment analysis model is less critical.
   • BUse publicly available social media data that is already anonymized, and test both the language model and the sentiment analysis model for adversarial text manipulations.
   • CUse a small sample of real citizen data, but only for the sentiment analysis model, to ensure accuracy.
   • DFocus on the infrastructure hosting the models, since data privacy is more important than model behavior.
   Reveal Solution Discussion Select an answer first
6. 35application · medium

   A red team is planning an exercise on an AI-based medical diagnosis support system. The exercise sponsor has stated that the team may not use any real patient data and must not test the system's behavior on rare diseases. The red team lead needs to create a scope document. What is the most important element to include?

   • AA clear list of exclusions (no real patient data, no rare disease testing) and assumptions about the test environment.
   • BA list of all team members and their roles.
   • CA detailed timeline for the exercise, including daily milestones.
   • DA description of the threat modeling framework that will be used.
   Reveal Solution Discussion Select an answer first