AI Security Testing, Evaluation, and Hardening COASP Practice Questions (Page 4)

Questions 16–20

1. 16foundation · easy

   What does the 'attack success rate' metric measure in AI security evaluation?

   • AThe percentage of training data that is correctly classified
   • BThe percentage of adversarial examples that successfully fool the model
   • CThe percentage of queries that the model can process per second
   • DThe percentage of model parameters that are vulnerable to extraction
   Reveal Solution Discussion Select an answer first
2. 17application · medium

   A multinational corporation is deploying an AI-based hiring tool that screens job applicants. The legal team has mandated that the tool must comply with the EU's General Data Protection Regulation (GDPR) and the upcoming EU AI Act. The security team is tasked with ensuring that the AI testing and hardening process aligns with these regulations. Which practice should the team prioritize to meet this compliance requirement?

   • AUse a third-party AI security testing tool that is certified by a recognized standards body to automatically ensure compliance.
   • BDocument the AI testing methodology, including adversarial testing results, and maintain a record of model versions and their associated risk assessments.
   • CFocus exclusively on technical hardening measures such as adversarial training and input sanitization to make the model robust.
   • DEnsure that all training data is stored in the same country where the model is deployed to comply with data residency requirements.
   Reveal Solution Discussion Select an answer first
4. 18expert · hard

   A defense contractor uses an AI model to detect objects in drone surveillance footage. The model was trained on a dataset that includes images of friendly and enemy vehicles. During a field test, the model misclassifies a friendly vehicle as an enemy vehicle when a specific sticker is placed on it. The team suspects an adversarial attack. The contractor needs to harden the model against this type of attack without retraining from scratch due to time constraints. Which defensive measure should the team implement?

   • AApply adversarial training using the specific sticker pattern as an adversarial example, and retrain the model on a dataset augmented with this pattern.
   • BImplement a detection mechanism that flags any image containing a sticker pattern as suspicious and routes it for manual review.
   • CUse a completely different model architecture that is known to be more robust to adversarial attacks, and retrain it from scratch.
   • DIncrease the model's confidence threshold to reduce the likelihood of misclassification, even if it increases the number of false negatives.
   Reveal Solution Discussion Select an answer first
5. 19application · medium

   A mobile banking app uses a deep learning model to detect fraudulent transactions. The security team wants to harden the model against evasion attacks that could allow fraudulent transactions to pass undetected. The team has a limited budget and needs to implement a defense that provides the best improvement in robustness for the effort. Which defensive measure should the team prioritize?

   • AUse input sanitization to remove any unusual patterns from transaction data before feeding it to the model.
   • BIncrease the model's complexity by adding more layers to improve its ability to distinguish between legitimate and fraudulent transactions.
   • CImplement adversarial training by augmenting the training data with adversarial examples generated using a fast gradient method.
   • DDeploy a complex ensemble of multiple models to make it harder for attackers to craft transferable adversarial examples.
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   What is the purpose of aligning AI security testing with industry standards and regulations?

   • ATo force organizations to use specific AI algorithms
   • BTo make AI systems more profitable by increasing their accuracy
   • CTo guarantee that AI systems are 100% secure against all attacks
   • DTo ensure that AI systems meet legal and regulatory requirements, such as data protection and safety standards
   Reveal Solution Discussion Select an answer first