A team ran experiments on two models for a summarization task and recorded BLEU scores: Model X scored 0.45, Model Y scored 0.52. Which model should they select based on this metric?

Cannot determine without more metrics

A data scientist is selecting a model for a legal document summarization task. They find a model on a hub with a model card that states: 'Trained on news articles and Wikipedia. Not recommended for legal or medical texts.' What should the data scientist do?

Ignore the model card and evaluate the model on a legal summarization test set.

Use the model but fine-tune it on a small set of legal documents.

Look for a model whose model card indicates training on legal corpora or domain adaptation.

Select the model anyway because it has high BLEU scores on general summarization benchmarks.

A developer is choosing an LLM for a medical Q&A application. They find a model card that states: 'Trained on general web text. Not intended for medical advice.' What should the developer do?

Find a model specifically trained on medical data

Fine-tune the model on a small medical dataset

Use the model because it has high general knowledge

Use the model and add a disclaimer

DatabricksCertified Generative AI Engineer Associate

Domain 3Objective 1

Model Selection and Evaluation GENERATIVE-AI-ENGINEER-ASSOCIATE Practice Questions (Page 4)

Part of the Section 3: Application Development domain, which makes up ~18% of our current practice bank. Databricks does not publish an official question count, but from its 90-minute exam (~35–60 total, ~6–11 in this domain), expect 1–2 from this objective — we provide 24 practice questions to prepare you well beyond it. (estimate)

24questions here

5free pages

6concepts

Questions 16–20

16foundation · easy
A team ran experiments on two models for a summarization task and recorded BLEU scores: Model X scored 0.45, Model Y scored 0.52. Which model should they select based on this metric?
Select an answer first
17application · medium
A data scientist is selecting a model for a legal document summarization task. They find a model on a hub with a model card that states: 'Trained on news articles and Wikipedia. Not recommended for legal or medical texts.' What should the data scientist do?
Select an answer first
18application · medium
A team is building a translation system and needs to compare two LLMs. They have a test set of English-French sentence pairs. Which metric is most appropriate for evaluating translation quality?
Select an answer first
19application · medium
A developer is choosing an LLM for a medical Q&A application. They find a model card that states: 'Trained on general web text. Not intended for medical advice.' What should the developer do?
Select an answer first
20expert · medium
A team has deployed a model and is now tracking its performance on live data. They notice that the model's accuracy is declining because the relationship between input features and the target variable has changed. What is this phenomenon?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to GENERATIVE-AI-ENGINEER-ASSOCIATE

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by Databricks. “GENERATIVE-AI-ENGINEER-ASSOCIATE” is a trademark of its owner, used for identification only.

Model Selection and Evaluation GENERATIVE-AI-ENGINEER-ASSOCIATE Practice Questions (Page 4)

Questions 16–20

A team ran experiments on two models for a summarization task and recorded BLEU scores: Model X scored 0.45, Model Y scored 0.52. Which model should they select based on this metric?

A data scientist is selecting a model for a legal document summarization task. They find a model on a hub with a model card that states: 'Trained on news articles and Wikipedia. Not recommended for legal or medical texts.' What should the data scientist do?

A team is building a translation system and needs to compare two LLMs. They have a test set of English-French sentence pairs. Which metric is most appropriate for evaluating translation quality?

A developer is choosing an LLM for a medical Q&A application. They find a model card that states: 'Trained on general web text. Not intended for medical advice.' What should the developer do?

A team has deployed a model and is now tracking its performance on live data. They notice that the model's accuracy is declining because the relationship between input features and the target variable has changed. What is this phenomenon?

Keep studying — Section 3: Application Development