AI System Hacking Methodologies, Frameworks, and Risk Implications COASP Practice Questions (Page 8)

Questions 36–40

1. 36application · medium

   A penetration tester is conducting an assessment of an AI-powered autonomous vehicle system. During the exploitation phase, the tester successfully causes the vehicle's object detection model to misidentify a stop sign as a speed limit sign. Which phase of the AI system hacking methodology does this action represent?

   • APost-exploitation, because the tester is assessing the impact of the attack.
   • BWeaponization, because the tester is crafting the adversarial image.
   • CExploitation, because the tester is actively delivering a malicious input to trigger a model failure.
   • DReconnaissance, because the tester is gathering information about the model's behavior.
   Reveal Solution Discussion Select an answer first
2. 37application · medium

   A security engineer is planning an offensive assessment of a company's customer-support chatbot. The engineer needs to systematically identify the chatbot's underlying model type, training data sources, and any exposed APIs before crafting any attacks. Which sequence of actions best follows the reconnaissance phase of an AI hacking methodology?

   • AImmediately send adversarial examples to the chatbot to test for misclassification, then document which inputs caused errors, then report the findings.
   • BReview the chatbot's privacy policy and terms of service to identify data retention practices, then interview the development team about model training, then test for prompt injection.
   • CEnumerate the network ports and services of the chatbot's hosting infrastructure, then run a vulnerability scanner against the web application, then attempt to extract the model weights.
   • DProbe the chatbot's responses with crafted inputs to infer model behavior, then search public code repositories and job postings for model and data details, then map the discovered APIs and endpoints.
   Reveal Solution Discussion Select an answer first
4. 38application · medium

   A healthcare organization deploys an AI system that predicts patient readmission risks. An attacker steals the model by querying it and reconstructing its decision boundaries. What is the most significant operational risk to the organization?

   • AThe organization faces legal penalties for failing to protect the model's intellectual property.
   • BThe attacker can craft inputs that cause the model to make incorrect predictions, potentially leading to misdiagnosis or inappropriate care.
   • CThe attacker gains access to the underlying patient data used for training.
   • DThe organization must shut down the AI system immediately to prevent further attacks.
   Reveal Solution Discussion Select an answer first
5. 39application · medium

   A red team is tasked with assessing an AI-powered fraud detection system. The team has completed the reconnaissance phase and identified that the model is a deep neural network trained on transaction data. They need to select a technique that will cause the model to misclassify fraudulent transactions as legitimate, while minimizing the chance of detection by the system's monitoring. Which MITRE ATLAS technique is most appropriate for this objective?

   • AAdversarial input manipulation, which crafts inputs that are imperceptibly altered to fool the model.
   • BModel inversion, which reconstructs training data from the model's outputs.
   • CModel stealing, which replicates the model's functionality by querying it.
   • DData poisoning, which injects malicious samples into the training set.
   Reveal Solution Discussion Select an answer first
6. 40foundation · easy

   What is a key ethical risk associated with adversarial manipulation of an AI system used in facial recognition?

   • AThe system's model weights may be copied, leading to intellectual property loss
   • BThe system's hardware may overheat and cause physical damage
   • CThe system may be tricked into misidentifying individuals, leading to false accusations or denial of services
   • DThe system's training data may be accidentally deleted, requiring re-collection
   Reveal Solution Discussion Select an answer first