Professional AI Red Team Reporting COASP Practice Questions (Page 1)

Questions 1–5

1. 1application · medium

   A red team report for a social media company identified a model poisoning vulnerability in the content recommendation system. The company wants to remediate the issue but has limited budget. Which recommendation should the report list as the top priority?

   • AIncrease the frequency of model retraining to reduce the window of exposure to poisoned data.
   • BImplement anomaly detection on training data to identify and remove poisoned samples before they affect the model.
   • CReplace the recommendation model with a completely new architecture that is inherently resistant to poisoning.
   • DDisable the recommendation system entirely until a full investigation is completed.
   Reveal Solution Discussion Select an answer first
2. 2application · medium

   In your AI red team report, you need to describe a model evasion attack that was successful against a facial recognition system. Which description is most appropriate for the 'Adversarial AI Specifics' section?

   • ARecommend that the client replace the facial recognition system entirely.
   • BExplain that the attack used adversarial perturbations to alter the model's classification, and discuss the potential for bypassing security controls.
   • CList the hardware specifications of the server hosting the model.
   • DFocus on the network vulnerabilities that allowed the attacker to access the model.
   Reveal Solution Discussion Select an answer first
4. 3expert · hard

   You are preparing the evidence for a red team report on a membership inference attack against a medical diagnosis model. The evidence includes the model's confidence scores for certain inputs, which could be used to infer whether a patient's data was in the training set. How should you present this evidence?

   • APresent the confidence scores in an aggregated or anonymized form, such as histograms or averages, and describe the attack methodology without revealing individual patient data.
   • BDescribe the attack in text only, without any quantitative evidence.
   • CProvide the raw model outputs for all test inputs, including any that contain personal information.
   • DInclude the exact confidence scores for each patient tested, along with their diagnosis codes.
   Reveal Solution Discussion Select an answer first
5. 4application · medium

   A red team completed an assessment of a university's AI-based admissions system. The most critical finding is that the system can be manipulated to favor certain applicants by submitting crafted application essays. The university's president, who is not technical, will read the executive summary. What should the executive summary emphasize?

   • AThe system's accuracy in predicting student success is 85%, which is above the industry average, so the overall risk is low.
   • BThe system's natural language processing model has a vulnerability in its attention mechanism that can be exploited with specific token sequences.
   • CThe system should be immediately taken offline until the vulnerability is fixed, to prevent any potential manipulation.
   • DThe admissions system could be manipulated to unfairly favor certain applicants, undermining the fairness and integrity of the admissions process.
   Reveal Solution Discussion Select an answer first
6. 5application · medium

   Your red team discovered that a customer-service chatbot, which is integrated with backend order systems, can be manipulated through prompt injection to reveal other customers' order details. You are assessing the risk for the report. Which factor is most important to consider when determining the business impact?

   • AThe chatbot's response time during the attack.
   • BThe complexity of the prompt injection payload used in the test.
   • CThe potential for unauthorized disclosure of personal data, which could lead to regulatory fines and loss of customer trust.
   • DThe number of red team members who participated in the test.
   Reveal Solution Discussion Select an answer first