AI Security Testing, Evaluation, and Hardening COASP Practice Questions (Page 7)

Questions 31–35

1. 31application · medium

   A company's AI-based chatbot is found to be leaking sensitive customer data when prompted with specific phrases. The incident response team needs to contain the issue while preserving the chatbot's availability for non-sensitive queries. Which action should be taken FIRST?

   • AImplement input filtering to block the known malicious prompt patterns
   • BDisconnect the chatbot's database to prevent data access
   • CTake the chatbot offline indefinitely for a full retraining
   • DIgnore the issue since it only affects a small number of prompts
   Reveal Solution Discussion Select an answer first
2. 32foundation · easy

   Which tool is specifically designed for generating adversarial examples to test AI models?

   • ACleverHans
   • BWireshark
   • CNmap
   • DMetasploit
   Reveal Solution Discussion Select an answer first
4. 33foundation · easy

   Which metric is specifically used to evaluate how well an AI model maintains its performance when subjected to adversarial perturbations?

   • ALatency
   • BMean squared error
   • CF1 score
   • DAdversarial robustness
   Reveal Solution Discussion Select an answer first
5. 34application · medium

   A healthcare AI vendor must comply with data protection regulations when testing models with patient data. The team wants to run adversarial simulations but is concerned about privacy. Which practice BEST aligns with compliance?

   • ASkip adversarial testing to avoid any data processing
   • BUse synthetic or de-identified data for adversarial testing whenever possible
   • CShare patient data with external researchers for broader testing
   • DRun all tests on production patient data to ensure accuracy
   Reveal Solution Discussion Select an answer first
6. 35expert · hard

   An e-commerce company's recommendation system has been manipulated by a competitor to promote their products. The attack was carried out by submitting fake user interactions to the system, causing it to learn biased recommendations. The security team has identified the attack as a form of data poisoning. The company wants to recover the system's integrity while minimizing disruption to the service. Which approach should the team take?

   • AApply adversarial training to the model to make it more robust to future poisoning attempts.
   • BIdentify and remove the poisoned data samples from the training set, then retrain the model on the cleaned data.
   • CDeploy the model immediately and monitor its recommendations for any further anomalies, assuming the attack will not recur.
   • DSwitch to a completely new model architecture and retrain from scratch using the original data, without investigating the poisoning.
   Reveal Solution Discussion Select an answer first