A team has collected SME feedback indicating that the agent's responses are often factually incorrect. They want to improve the agent. They have a custom Scorer that checks factual consistency with a provided context. What is the best approach to improve the agent and measure the improvement?

Change the underlying LLM to a larger model that might be more factually accurate.

Ignore the SME feedback and rely on the custom Scorer to identify issues.

Use the SME feedback to refine the agent's prompt to emphasize factual accuracy, then re-evaluate using the custom Scorer to measure the improvement.

Use MLflow tracing to manually inspect the responses after the prompt change.

DatabricksCertified Generative AI Engineer Associate

Domain 6Objective 3

Agent Evaluation and Feedback GENERATIVE-AI-ENGINEER-ASSOCIATE Practice Questions (Page 6)

Part of the Section 6: Evaluation and Monitoring domain, which makes up ~17% of our current practice bank. Databricks does not publish an official question count, but from its 90-minute exam (~35–60 total, ~6–10 in this domain), expect 1–2 from this objective — we provide 26 practice questions to prepare you well beyond it. (estimate)

26questions here

6free pages

6concepts

Questions 26–26

26expert · medium
A team has collected SME feedback indicating that the agent's responses are often factually incorrect. They want to improve the agent. They have a custom Scorer that checks factual consistency with a provided context. What is the best approach to improve the agent and measure the improvement?
Select an answer first

Finished these 1 questions?

Review the revealed explanations, or continue through the curriculum.

No more pagesBack to GENERATIVE-AI-ENGINEER-ASSOCIATE

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by Databricks. “GENERATIVE-AI-ENGINEER-ASSOCIATE” is a trademark of its owner, used for identification only.

Agent Evaluation and Feedback GENERATIVE-AI-ENGINEER-ASSOCIATE Practice Questions (Page 6)

Questions 26–26

A team has collected SME feedback indicating that the agent's responses are often factually incorrect. They want to improve the agent. They have a custom Scorer that checks factual consistency with a provided context. What is the best approach to improve the agent and measure the improvement?

Keep studying — Section 6: Evaluation and Monitoring