A team has implemented an input guardrail that uses a classifier to detect harmful prompts. They want to evaluate its effectiveness before deployment. They have a labeled dataset of prompts, but it is imbalanced, with very few harmful examples. Which approach is most appropriate for evaluating the guardrail?

Use F1 score, but only on the majority class.

Use a confusion matrix, but only report true negatives.

Use accuracy as the primary metric, as it is easy to understand.

Use precision and recall metrics, focusing on recall for harmful prompts to minimize missed detections.

A company is deploying an LLM-powered assistant that handles both internal and external queries. They want to implement guardrails that prevent the assistant from leaking confidential internal information. Which combination of guardrails is most appropriate?

Input guardrail that blocks prompts asking for confidential information, and output guardrail that redacts confidential data from responses.

A system prompt that instructs the model not to reveal confidential information.

Only an output guardrail that redacts confidential data from responses.

Only an input guardrail that blocks prompts asking for confidential information.

Which of the following is an example of a qualitative evaluation technique for LLM responses?

Measuring the number of tokens generated per second

Using a rubric to score responses for clarity and appropriateness

Calculating perplexity of the generated text

Comparing the output to a reference using cosine similarity

A team is building a multilingual chatbot. They want to implement an input guardrail that blocks harmful prompts in multiple languages. However, they have limited resources and cannot train a custom model for each language. Which approach is most effective?

Use a keyword-based guardrail with keywords from each language.

Apply the output guardrail only, as it can catch harmful content regardless of input language.

Use a pre-trained multilingual moderation model that can detect harmful content across languages.

Translate all prompts to English and then apply an English-only guardrail.

During a qualitative evaluation of an LLM-based therapy chatbot, a reviewer notices that the model sometimes responds to users in a way that could be interpreted as encouraging self-harm. Which type of issue is this, and what guardrail is most appropriate?

Incoherence; implement a system prompt that instructs the model to be supportive.

Harmful content; implement an output guardrail that detects and blocks responses related to self-harm.

Irrelevance; implement a post-processing step that checks for topic alignment.

Bias; implement an input guardrail that filters out prompts about mental health.

DatabricksCertified Generative AI Engineer Associate

Domain 3Objective 4

Safety and Guardrails GENERATIVE-AI-ENGINEER-ASSOCIATE Practice Questions (Page 5)

Part of the Section 3: Application Development domain, which makes up ~18% of our current practice bank. Databricks does not publish an official question count, but from its 90-minute exam (~35–60 total, ~6–11 in this domain), expect 1–2 from this objective — we provide 31 practice questions to prepare you well beyond it. (estimate)

31questions here

7free pages

7concepts

Questions 21–25

21expert · hard
A team has implemented an input guardrail that uses a classifier to detect harmful prompts. They want to evaluate its effectiveness before deployment. They have a labeled dataset of prompts, but it is imbalanced, with very few harmful examples. Which approach is most appropriate for evaluating the guardrail?
Select an answer first
22application · medium
A company is deploying an LLM-powered assistant that handles both internal and external queries. They want to implement guardrails that prevent the assistant from leaking confidential internal information. Which combination of guardrails is most appropriate?
Select an answer first
23foundation · easy
Which of the following is an example of a qualitative evaluation technique for LLM responses?
Select an answer first
24expert · hard
A team is building a multilingual chatbot. They want to implement an input guardrail that blocks harmful prompts in multiple languages. However, they have limited resources and cannot train a custom model for each language. Which approach is most effective?
Select an answer first
25application · medium
During a qualitative evaluation of an LLM-based therapy chatbot, a reviewer notices that the model sometimes responds to users in a way that could be interpreted as encouraging self-harm. Which type of issue is this, and what guardrail is most appropriate?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to GENERATIVE-AI-ENGINEER-ASSOCIATE

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by Databricks. “GENERATIVE-AI-ENGINEER-ASSOCIATE” is a trademark of its owner, used for identification only.