AI System Hacking Methodologies, Frameworks, and Risk Implications COASP Practice Questions (Page 4)

Questions 16–20

1. 16expert · hard

   A red team is using MITRE ATLAS to plan an attack on a company's AI-powered recommendation system. They need to choose a technique that will have the highest impact while minimizing the chance of detection. The team has access to the system's training data pipeline but not the model itself. Which technique should they prioritize?

   • APerforming a model inversion attack to extract sensitive user data from the model.
   • BExploiting a vulnerability in the web application to gain direct access to the model's server.
   • CPoisoning the training data by injecting malicious samples that cause the model to recommend a competitor's products.
   • DConducting a denial-of-service attack on the recommendation system's API.
   Reveal Solution Discussion Select an answer first
2. 17application · medium

   A company's AI-powered recruitment tool is found to be biased against candidates from certain universities. An investigation reveals that an attacker manipulated the training data to include biased hiring decisions. What is the most appropriate immediate response to mitigate the business risk?

   • ARetrain the model on a clean, validated dataset and implement data integrity controls.
   • BContinue using the model but add a disclaimer that it may be biased.
   • CDelete the model and stop using AI for recruitment entirely.
   • DOnly use the model for candidates from universities that were not affected by the bias.
   Reveal Solution Discussion Select an answer first
4. 18foundation · easy

   How does MITRE ATLAS relate to traditional hacking methodologies like the Cyber Kill Chain?

   • AIt replaces the Cyber Kill Chain entirely by focusing only on AI-specific attacks
   • BIt is unrelated to traditional hacking methodologies because AI attacks are fundamentally different
   • CIt only covers post-exploitation activities and ignores early phases like weaponization
   • DIt maps AI-specific attack techniques to the phases of the Cyber Kill Chain, such as reconnaissance and exploitation
   Reveal Solution Discussion Select an answer first
5. 19application · medium

   A bank uses an AI model to detect fraudulent transactions. An attacker successfully performs a model extraction attack, creating a copy of the model. Which ethical risk is MOST directly associated with this attack?

   • AThe attacker may expose the bank's customer data because the model contains raw transaction records.
   • BThe attacker may use the copied model to identify vulnerabilities in the bank's fraud detection, enabling more effective fraud.
   • CThe bank may be forced to shut down because the model is no longer proprietary.
   • DThe bank's employees may lose their jobs because the model is now publicly available.
   Reveal Solution Discussion Select an answer first
6. 20application · medium

   A security consultant is hired to assess an AI-based credit-scoring system. During the reconnaissance phase, the consultant discovers that the model's training data includes publicly available financial records and that the model is hosted behind an API that returns confidence scores. The consultant wants to map the attack surface systematically and identify which adversarial techniques are most relevant before conducting any active exploitation. Which approach best aligns with the AI system hacking methodology?

   • AFocus only on the API's rate-limiting controls, because the main risk is denial of service rather than model manipulation.
   • BUse MITRE ATLAS to map the discovered API and data exposure to adversarial techniques, then prioritize techniques that target the model's training data and inference interface.
   • CImmediately attempt to extract the model by sending thousands of queries to the API, since reconnaissance is complete once the API is identified.
   • DApply a traditional network penetration testing framework, such as PTES, and ignore AI-specific techniques because the model is just another web service.
   Reveal Solution Discussion Select an answer first