AI and Machine Learning Fundamentals from an Offensive Security Perspective COASP Practice Questions (Page 3)

Questions 11–15

1. 11application · easy

   A security team is reviewing the security of a machine learning model that classifies images for a social media platform. The team discovers that an attacker has been submitting specially crafted images that are misclassified, causing inappropriate content to be posted. The attacker does not have access to the model's parameters or training data. Which type of adversarial attack is the attacker most likely performing?

   • AModel extraction attack — querying the model to replicate its functionality.
   • BData inference attack — using the model to infer private training data.
   • CEvasion attack — crafting inputs that cause misclassification at inference time.
   • DPoisoning attack — injecting malicious data into the training set.
   Reveal Solution Discussion Select an answer first
2. 12application · medium

   A security architect is designing a threat model for a machine learning system that uses a third-party pre-trained model. The system is deployed on a Kubernetes cluster and accesses data from a database. The architect wants to identify the component that is most susceptible to a poisoning attack. Which component should the architect focus on?

   • AThe model's inference API endpoint.
   • BThe training data pipeline that ingests data from external sources.
   • CThe database that stores the training data.
   • DThe Kubernetes cluster's network policies.
   Reveal Solution Discussion Select an answer first
4. 13application · medium

   A red team is tasked with assessing the security of an AI-based intrusion detection system (IDS). The red team has already mapped the system's components and identified that the IDS uses a supervised model trained on network traffic data. The red team now wants to test whether they can evade detection by crafting network traffic that the model misclassifies as benign. Which step of the hacking methodology does this testing represent?

   • AReconnaissance — scanning the network to identify hosts.
   • BPost-exploitation — maintaining access to the network.
   • CReporting — documenting the findings.
   • DExploitation — crafting adversarial network traffic to bypass the IDS.
   Reveal Solution Discussion Select an answer first
5. 14expert · hard

   A security team is defending a machine learning model that classifies medical images. The team has implemented adversarial training to improve robustness against evasion attacks. However, they are now concerned that the model is vulnerable to poisoning attacks. The team has limited resources and must choose between two defenses: (1) implementing robust aggregation techniques during training, or (2) adding a validation layer that checks input data for anomalies before inference. Which defense is more appropriate for mitigating poisoning attacks?

   • ANeither is effective; the team should retrain the model from scratch.
   • BBoth are equally effective for poisoning attacks.
   • CInput validation layer, because it detects malicious inputs before they reach the model.
   • DRobust aggregation techniques, because they reduce the impact of malicious training samples on the model.
   Reveal Solution Discussion Select an answer first
6. 15foundation · easy

   What is the primary goal of an evasion attack on a machine learning model?

   • ATo craft inputs that are misclassified by the model while appearing normal to humans.
   • BTo corrupt the training data so the model learns incorrect behavior.
   • CTo steal the model's architecture and weights by querying it.
   • DTo overload the model's API with requests to cause denial of service.
   Reveal Solution Discussion Select an answer first