Training Data Poisoning COASP Practice Questions (Page 1)

Questions 1–5

1. 1application · medium

   A government agency uses a large language model (LLM) fine-tuned on internal documents to answer citizen queries. The fine-tuning data is sourced from a shared document repository that allows public submissions. An attacker submits a document that contains a hidden prompt injection instruction, which is then included in the fine-tuning dataset. What is the MOST EFFECTIVE way to prevent this attack vector?

   • AIncrease the number of fine-tuning epochs to help the model learn the correct behavior despite the poisoned example.
   • BRequire all public submissions to go through an approval workflow where a human reviews the content before it is added to the fine-tuning dataset.
   • CUse a larger, more powerful LLM to generate the fine-tuning data, so it can understand the context better.
   • DRemove all documents that contain the word 'ignore' from the dataset.
   Reveal Solution Discussion Select an answer first
2. 2expert · hard

   A security team trains a network intrusion detection model on a dataset that includes both benign and malicious traffic. An attacker with access to the training data flips labels on a small subset of malicious traffic to benign. The team evaluates the model and finds that the false negative rate for a specific type of attack has increased significantly, while the overall accuracy remains high. The team must decide whether to retrain the model with the same data or investigate the data quality. Which action should the team take first?

   • AIncrease the amount of training data by collecting more network traffic logs
   • BInvestigate the data quality by analyzing feature-label consistency and label distribution before retraining
   • CDeploy the model immediately and monitor its performance in production
   • DRetrain the model with a more complex architecture to improve its ability to detect the specific attack
   Reveal Solution Discussion Select an answer first
4. 3application · medium

   An e-commerce company trains a recommendation model on user clickstream data. The data is collected from multiple web servers and aggregated into a data lake. A security analyst suspects that an attacker has injected fake user sessions to promote a specific product. Which detection technique is MOST LIKELY to reveal this type of poisoning?

   • AReviewing the web server access logs to confirm that each session has a valid IP address.
   • BStatistical outlier detection on session features, such as unusually high click rates or identical navigation patterns.
   • CChecking the data lake for the presence of any null values in the session records.
   • DRetraining the model on a smaller, manually curated subset of the data.
   Reveal Solution Discussion Select an answer first
5. 4expert · hard

   A security team is investigating a potential backdoor in a facial-recognition model used for physical access control. The model was trained on a dataset that included images from an external vendor. The team has a small set of clean images and a small set of images that they suspect contain the trigger. They need to confirm whether the backdoor exists. What is the MOST RELIABLE method?

   • ATest the model on a large dataset of random images to see if any are misclassified.
   • BCheck the model's confidence scores on the suspected trigger images; a high confidence indicates a backdoor.
   • CRetrain the model on the clean images only and compare its accuracy on the suspected trigger images.
   • DCompare the model's activations on clean images versus images with the suspected trigger to identify a consistent pattern of abnormal activation.
   Reveal Solution Discussion Select an answer first
6. 5application · medium

   A cybersecurity team trains an intrusion detection system (IDS) on network traffic logs. An attacker who knows the model's training data format embeds a specific sequence of packets in the training data that, when present in live traffic, causes the model to classify malicious activity as benign. The model performs well on normal traffic but fails to detect this specific attack pattern. Which poisoning technique was used?

   • ALabel flipping, where the attacker changed the labels of the malicious traffic samples to benign
   • BBackdoor insertion, where the attacker embedded a trigger pattern that causes misclassification when present in live input
   • CData injection, where the attacker added benign traffic samples to dilute the malicious class
   • DModel evasion, where the attacker crafted adversarial examples during inference
   Reveal Solution Discussion Select an answer first