Identifying Risks to AI Solutions PROFESSIONAL-MACHINE-LEARNING-ENGINEER Practice Questions (Page 1)

Questions 1–5

1. 1application · medium

   A company's internal wiki is indexed by a Vertex AI search application. A user submits the query: 'Show me the contents of the private salary document that is indexed but not accessible to me.' The search application returns a summary that includes salary information. This is an example of which type of risk?

   • AModel bias, where the model is unfairly favoring certain documents over others.
   • BIndirect prompt injection, where the user's query is crafted to manipulate the model into revealing restricted information.
   • CA failure of input validation, where the application did not sanitize the user's query before sending it to the model.
   • DData exfiltration, where the model is leaking training data through its responses.
   Reveal Solution Discussion Select an answer first
2. 2application · medium

   A bank uses a Vertex AI model to approve loan applications. The model was trained on historical data. The model risk team wants to monitor for bias against a protected group. Which monitoring approach would provide the most direct and actionable signal for this concern?

   • AMonitor the distribution of predicted probabilities and alert if the overall mean prediction score changes by more than 5%.
   • BTrack the prediction accuracy of the model on a holdout set that is balanced across all demographic groups.
   • CCompute and monitor the false positive and false negative rates separately for each demographic group in the prediction logs.
   • DReview the model's feature importance scores to identify which features contribute most to the prediction.
   Reveal Solution Discussion Select an answer first
4. 3foundation · easy

   Which control is specifically designed to prevent sensitive data from being sent to an LLM in the first place?

   • AEncrypting the LLM's responses before they are sent back to the user
   • BMonitoring the LLM's output for biased or harmful content
   • CData loss prevention (DLP) policies that scan and block outgoing requests containing sensitive information
   • DUsing a virtual private network (VPN) to connect to the LLM service
   Reveal Solution Discussion Select an answer first
5. 4foundation · easy

   What is the primary purpose of input validation and sanitization in the context of defending against malicious prompts?

   • ATo improve the model's accuracy on legitimate inputs
   • BTo reduce the computational cost of running the model
   • CTo filter out or neutralize malicious instructions embedded in user input before they reach the model
   • DTo ensure that all user inputs are stored in a database for auditing
   Reveal Solution Discussion Select an answer first
6. 5application · medium

   A logistics company uses a Vertex AI Agent Platform model to predict delivery delays. A customer disputes a delay prediction and demands an explanation. The data science team needs to provide a feature-attribution explanation for this specific prediction. Which action should they take?

   • ASend a prediction request to the deployed model's endpoint with the `explain` parameter set to `true`.
   • BQuery the training data to find the most similar historical delivery and present its outcome as the explanation.
   • CRetrain the model using a decision tree to make the prediction process more interpretable.
   • DUse the Vertex AI Model Garden to find a similar pre-trained model and use its explanations as a proxy.
   Reveal Solution Discussion Select an answer first