AI Security Testing, Evaluation, and Hardening COASP Practice Questions (Page 9)

Questions 41–45

1. 41application · medium

   A company is planning its first AI security test for a recommendation system. The team has limited resources and wants to focus on the most likely attack vectors. Which approach best defines the scope of the test?

   • AFocus only on attacks that have been reported in the news
   • BTest all possible attack types equally to ensure complete coverage
   • CPrioritize attacks based on the system's threat model, including evasion and extraction risks
   • DSkip the scope definition and test the model's accuracy on a standard benchmark
   Reveal Solution Discussion Select an answer first
2. 42application · medium

   A security team is evaluating the robustness of a face-recognition model used for physical access control. They have run a series of evasion attacks and need to decide whether the model is safe to deploy. The team's primary concern is that an attacker should not be able to impersonate a legitimate user. Which metric would be most appropriate to prioritize in this evaluation?

   • AThe model's training time, to ensure it was trained efficiently and within the project's budget.
   • BAttack success rate (ASR) at a fixed perturbation budget, measuring how often adversarial examples bypass the model.
   • CThe model's clean accuracy on a standard benchmark dataset such as LFW (Labeled Faces in the Wild).
   • DThe model's inference latency, ensuring it can process access requests quickly enough for real-time use.
   Reveal Solution Discussion Select an answer first
4. 43expert · hard

   During a routine security test, a team discovers that an AI model has been silently poisoned over several months, causing subtle biases in loan approvals. The model is in production and used by multiple departments. The team must respond while minimizing disruption. Which action should be the FIRST step in the incident response plan?

   • AImmediately delete the poisoned model and deploy a new one from scratch
   • BNotify all departments that the model is compromised and halt all loan approvals
   • CIgnore the poisoning since the model still works for most cases
   • DIsolate the model from production traffic and route requests to a shadow model for evaluation
   Reveal Solution Discussion Select an answer first
5. 44foundation · easy

   Which hardening technique involves augmenting the training dataset with adversarial examples to improve model robustness?

   • AGradient clipping
   • BAdversarial training
   • CModel pruning
   • DInput sanitization
   Reveal Solution Discussion Select an answer first
6. 45application · medium

   A security team needs to automate the generation of adversarial examples for a natural language processing model that detects phishing emails. They want to test the model's robustness against synonym substitution and character-level perturbations. Which tool is MOST appropriate for this task?

   • ACleverHans, which focuses on image-based adversarial attacks
   • BA custom script using random word replacement without semantic constraints
   • CTextAttack, which provides built-in transformations for synonym swaps and character-level changes
   • DART (Adversarial Robustness Toolbox), which supports only tabular data attacks
   Reveal Solution Discussion Select an answer first