What does the ROUGE metric primarily measure in text summarization evaluation?

Semantic similarity using contextual embeddings

The precision of n-gram matches in machine translation

The overlap of n-grams between the generated summary and reference summaries

The fluency of the generated text as judged by humans

A financial services firm has built a multi-step workflow that uses a foundation model to extract data from forms, validate it, and generate a report. They want to evaluate the workflow's overall reliability. Which approach should they use?

Measure end-to-end task completion rate and error rates at each step

Use user satisfaction surveys only

Use BLEU to evaluate the final report

Use ROUGE to compare reports to references

A healthcare company is evaluating a foundation model that generates patient education materials. They need to ensure the outputs are medically accurate, safe, and empathetic. They are concerned that automated metrics might miss subtle safety issues. Which evaluation method should they prioritize?

Human-in-the-loop evaluation with domain experts

ROUGE score on reference materials

BLEU score on translated materials

AWSCertified AI Practitioner

Domain 3Objective 4

Task Statement 3.4: Describe Methods to Evaluate FM Performance. AIF-C01 Practice Questions (Page 4)

Part of the Content Domain 3: Applications of Foundation Models domain, which makes up ~32% of our current practice bank. AWS does not publish an official question count, but from its 90-minute exam (~35–60 total, ~11–19 in this domain), expect 3–5 from this objective — we provide 40 practice questions to prepare you well beyond it. (estimate)

40questions here

8free pages

14concepts

Questions 16–20

16foundation · easy
What does the ROUGE metric primarily measure in text summarization evaluation?
Select an answer first
17foundation · easy
Which of the following is a common variant of the ROUGE metric?
Select an answer first
18foundation · easy
Which of the following is a standard benchmark dataset used to evaluate foundation models on language understanding tasks?
Select an answer first
19application · medium
A financial services firm has built a multi-step workflow that uses a foundation model to extract data from forms, validate it, and generate a report. They want to evaluate the workflow's overall reliability. Which approach should they use?
Select an answer first
20application · medium
A healthcare company is evaluating a foundation model that generates patient education materials. They need to ensure the outputs are medically accurate, safe, and empathetic. They are concerned that automated metrics might miss subtle safety issues. Which evaluation method should they prioritize?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to AIF-C01

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by AWS. “AIF-C01” is a trademark of its owner, used for identification only.

Task Statement 3.4: Describe Methods to Evaluate FM Performance. AIF-C01 Practice Questions (Page 4)

Questions 16–20

What does the ROUGE metric primarily measure in text summarization evaluation?

Which of the following is a common variant of the ROUGE metric?

Which of the following is a standard benchmark dataset used to evaluate foundation models on language understanding tasks?

A financial services firm has built a multi-step workflow that uses a foundation model to extract data from forms, validate it, and generate a report. They want to evaluate the workflow's overall reliability. Which approach should they use?

A healthcare company is evaluating a foundation model that generates patient education materials. They need to ensure the outputs are medically accurate, safe, and empathetic. They are concerned that automated metrics might miss subtle safety issues. Which evaluation method should they prioritize?

Keep studying — Content Domain 3: Applications of Foundation Models