AI Security Testing, Evaluation, and Hardening COASP Practice Questions (Page 1)

Questions 1–5

1. 1foundation · easy

   Why do incident response plans for AI systems need to include AI-specific considerations?

   • ABecause AI systems are always cloud-based and require cloud-specific response procedures
   • BBecause AI incidents may involve model poisoning, adversarial attacks, or data leakage that require specialized detection and response techniques
   • CBecause AI incidents are always caused by external attackers
   • DBecause AI systems cannot be monitored using traditional security tools
   Reveal Solution Discussion Select an answer first
2. 2foundation · easy

   In adversarial attack simulation, what is the primary objective of an evasion attack?

   • ATo extract the model's parameters or training data through queries
   • BTo overwhelm the model with a high volume of requests to cause denial of service
   • CTo corrupt the training data so the model learns incorrect patterns
   • DTo cause the model to misclassify a malicious input as benign
   Reveal Solution Discussion Select an answer first
4. 3expert · hard

   A healthcare organization has deployed an AI model that predicts patient readmission risk. During a routine audit, the security team discovers that the model has been systematically misclassifying a specific demographic group, leading to incorrect treatment recommendations. The team suspects that the model was poisoned during training. The organization must respond quickly to mitigate patient harm while preserving the model's overall utility. Which action should the team take first?

   • AQuarantine the affected model and switch to a rule-based fallback system for readmission predictions while investigating the poisoning.
   • BRetrain the model from scratch using the original training data, assuming the poisoning was introduced during the current training run.
   • CApply adversarial training to the existing model to make it more robust to the observed misclassifications.
   • DIncrease the model's decision threshold to reduce the number of false positives, which may mitigate the misclassifications.
   Reveal Solution Discussion Select an answer first
5. 4application · medium

   A company's AI model for detecting fraudulent transactions is being attacked with adversarial inputs that cause false negatives. The team wants to implement a defense that can be updated quickly as new attack patterns emerge. Which approach is MOST suitable?

   • ARetrain the model from scratch every time a new attack is discovered
   • BDeploy a rule-based input filter that can be updated with new attack signatures
   • CDisable the model until the attack is fully understood
   • DIncrease the model's decision threshold to reduce false negatives
   Reveal Solution Discussion Select an answer first
6. 5application · medium

   A pharmaceutical company is developing an AI model to predict drug interactions. The model will be used in clinical decision support, and the company must comply with FDA regulations for software as a medical device (SaMD). The security team is responsible for ensuring that the AI testing process meets regulatory requirements. Which practice should the team implement to align with FDA expectations?

   • AMaintain a comprehensive validation and verification (V&V) report that documents the AI model's performance, including robustness testing under adversarial conditions.
   • BFocus on the model's accuracy on a standard drug-interaction dataset, as this is the primary metric required by the FDA.
   • CUse an open-source AI security testing tool to automatically generate a compliance report, without needing to document the testing process manually.
   • DEnsure that the AI model is deployed on a cloud platform that is certified for healthcare data, such as AWS or Azure.
   Reveal Solution Discussion Select an answer first