A data science team is debugging why their agent sometimes returns incorrect answers. They suspect the issue is in the tool-calling sequence. They have already instrumented the agent with MLflow tracing. What should they do to identify the problematic step?

Use mlflow.evaluate() to compute a metric that measures the correctness of each tool call.

Inspect the MLflow trace spans to see the sequence of tool calls, inputs, and outputs.

Review the agent's prompt template to see if the instructions are ambiguous.

Check the model registry for the latest version of the agent's model.

What is the primary goal of collecting subject matter expert (SME) feedback on agent responses?

To identify areas where the agent can be improved

To increase the agent's response speed

To replace automated evaluation metrics

To reduce the cost of running the agent

Which of the following is a valid way to incorporate SME feedback into agent improvement?

Updating the evaluation criteria to reflect SME insights

Deleting all feedback after a review

Storing feedback in a separate system without using it

Automatically rejecting any feedback that is negative

DatabricksCertified Generative AI Engineer Associate

Domain 6Objective 3

Agent Evaluation and Feedback GENERATIVE-AI-ENGINEER-ASSOCIATE Practice Questions (Page 1)

Part of the Section 6: Evaluation and Monitoring domain, which makes up ~17% of our current practice bank. Databricks does not publish an official question count, but from its 90-minute exam (~35–60 total, ~6–10 in this domain), expect 1–2 from this objective — we provide 26 practice questions to prepare you well beyond it. (estimate)

26questions here

6free pages

6concepts

Questions 1–5

1foundation · easy
Which Databricks feature allows you to implement custom evaluation metrics for LLM and agent outputs?
Select an answer first
2application · medium
A data science team is debugging why their agent sometimes returns incorrect answers. They suspect the issue is in the tool-calling sequence. They have already instrumented the agent with MLflow tracing. What should they do to identify the problematic step?
Select an answer first
3foundation · easy
What is the primary goal of collecting subject matter expert (SME) feedback on agent responses?
Select an answer first
4foundation · easy
Which MLflow feature is used to inspect the detailed execution flow of an agent, including tool calls and intermediate steps?
Select an answer first
5foundation · easy
Which of the following is a valid way to incorporate SME feedback into agent improvement?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to GENERATIVE-AI-ENGINEER-ASSOCIATE

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by Databricks. “GENERATIVE-AI-ENGINEER-ASSOCIATE” is a trademark of its owner, used for identification only.

Agent Evaluation and Feedback GENERATIVE-AI-ENGINEER-ASSOCIATE Practice Questions (Page 1)

Questions 1–5

Which Databricks feature allows you to implement custom evaluation metrics for LLM and agent outputs?

A data science team is debugging why their agent sometimes returns incorrect answers. They suspect the issue is in the tool-calling sequence. They have already instrumented the agent with MLflow tracing. What should they do to identify the problematic step?

What is the primary goal of collecting subject matter expert (SME) feedback on agent responses?

Which MLflow feature is used to inspect the detailed execution flow of an agent, including tool calls and intermediate steps?

Which of the following is a valid way to incorporate SME feedback into agent improvement?

Keep studying — Section 6: Evaluation and Monitoring