Scoping an AI Red Team Exercise and Threat Modeling COASP Practice Questions (Page 6)

Questions 26–30

1. 26application · medium

   A red team is about to start an exercise on an AI-powered resume screening tool. The exercise sponsor wants to ensure that the team does not access the production database and does not test the model's performance on protected attributes like race or gender. What should the red team lead do first?

   • ADocument these constraints as explicit exclusions and assumptions in the exercise scope document, and share it with the sponsor for approval.
   • BAsk the sponsor to provide a written list of allowed test cases before the exercise begins.
   • CInclude the production database in the exercise but only read a small sample, to improve realism.
   • DProceed with the exercise and avoid those areas informally, since the sponsor's intent is clear.
   Reveal Solution Discussion Select an answer first
2. 27application · medium

   A red team lead is documenting the scope for an AI red team exercise on a healthcare diagnostic model. The documentation must be shared with both technical staff and hospital administrators. Which documentation approach best serves this audience?

   • AA document that lists only the exclusions and constraints, without detailing the in-scope assets or objectives.
   • BA single technical document that details every attack vector, tool, and exploit used, with no executive summary.
   • CA document with an executive summary of objectives and constraints, followed by a technical appendix with detailed attack surfaces and assumptions.
   • DA verbal briefing to administrators and a separate email to technical staff, with no written documentation.
   Reveal Solution Discussion Select an answer first
4. 28foundation · easy

   During the scoping phase of an AI red team exercise, which of the following is the primary purpose of defining the exercise's boundaries?

   • ATo guarantee that no real user data is accessed during the exercise
   • BTo establish which components, environments, and behaviors are in scope and which are excluded from testing
   • CTo ensure the red team has unlimited access to all systems for thorough testing
   • DTo document the red team's internal communication protocols
   Reveal Solution Discussion Select an answer first
5. 29application · medium

   A security team is using STRIDE to threat model an AI-based image recognition system used for access control. They are analyzing the model's inference API. Which threat is best categorized under the 'Tampering' element of STRIDE?

   • AAn attacker spoofs the API endpoint to trick the system into accepting false inputs.
   • BAn attacker sends a large number of requests to the API to exhaust resources.
   • CAn attacker modifies the input image to cause the model to recognize an unauthorized person as authorized.
   • DAn attacker reads the model's confidence scores to infer information about the training data.
   Reveal Solution Discussion Select an answer first
6. 30expert · hard

   A red team is documenting the scope for an AI red team exercise. The exercise involves an AI system that uses a third-party API for part of its functionality. The sponsor has stated that the third-party API is out of scope. However, the red team discovers that the AI system sends sensitive data to the third-party API. What should the red team do?

   • AAsk the sponsor to change the scope to include the third-party API before proceeding.
   • BDocument this discovery as a finding and recommend a separate assessment of the third-party API, while keeping it out of scope for the current exercise.
   • CIgnore the data flow to the third-party API, since it is out of scope.
   • DInclude the third-party API in the exercise anyway, because the data flow is a critical risk.
   Reveal Solution Discussion Select an answer first