A team is building a customer-support chatbot that answers questions from a fixed knowledge base. They have a labeled dataset of 1,000 question-answer pairs. They want to evaluate the chatbot's responses for factual correctness. Which evaluation approach is most appropriate?

Use ROUGE to measure the overlap between the chatbot's response and the expected answer.

Use a reference-free LLM-as-judge that scores the response for helpfulness without comparing to a reference.

Use exact match to compare the chatbot's response to the expected answer.

Use an LLM-as-judge with reference answers to compare the chatbot's response against the expected answer.

A data science team is evaluating a generative model that produces summaries of news articles. They do not have any reference summaries. They want to assess whether the summaries are coherent and contain no hallucinations. Which evaluation approach should they use?

Use a reference-free LLM-as-judge to score the summaries for coherence and factual consistency with the source articles.

Use exact match to compare the generated summaries to the original articles.

Use ROUGE to compare the generated summaries against the source articles.

Use BLEU to compare the generated summaries against a set of human-written reference summaries.

DatabricksCertified Generative AI Engineer Associate

Domain 6Objective 5

Evaluation Judges and Ground Truth GENERATIVE-AI-ENGINEER-ASSOCIATE Practice Questions (Page 4)

Part of the Section 6: Evaluation and Monitoring domain, which makes up ~17% of our current practice bank. Databricks does not publish an official question count, but from its 90-minute exam (~35–60 total, ~6–10 in this domain), expect 1–2 from this objective — we provide 17 practice questions to prepare you well beyond it. (estimate)

17questions here

4free pages

4concepts

Questions 16–17

16application · medium
A team is building a customer-support chatbot that answers questions from a fixed knowledge base. They have a labeled dataset of 1,000 question-answer pairs. They want to evaluate the chatbot's responses for factual correctness. Which evaluation approach is most appropriate?
Select an answer first
17application · medium
A data science team is evaluating a generative model that produces summaries of news articles. They do not have any reference summaries. They want to assess whether the summaries are coherent and contain no hallucinations. Which evaluation approach should they use?
Select an answer first

Finished these 2 questions?

Review the revealed explanations, or continue through the curriculum.

No more pagesBack to GENERATIVE-AI-ENGINEER-ASSOCIATE

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by Databricks. “GENERATIVE-AI-ENGINEER-ASSOCIATE” is a trademark of its owner, used for identification only.

Evaluation Judges and Ground Truth GENERATIVE-AI-ENGINEER-ASSOCIATE Practice Questions (Page 4)

Questions 16–17

A team is building a customer-support chatbot that answers questions from a fixed knowledge base. They have a labeled dataset of 1,000 question-answer pairs. They want to evaluate the chatbot's responses for factual correctness. Which evaluation approach is most appropriate?

A data science team is evaluating a generative model that produces summaries of news articles. They do not have any reference summaries. They want to assess whether the summaries are coherent and contain no hallucinations. Which evaluation approach should they use?

Keep studying — Section 6: Evaluation and Monitoring