When evaluating an agent that uses tools, what does 'tool use correctness' assess?

Whether the agent responds quickly

Whether the agent has access to many tools

Whether the agent selects and uses the appropriate tool correctly

Whether the agent's responses are grammatically correct

Why is cost per interaction an important metric for business evaluation of AI applications?

It reflects the model's accuracy

It helps determine the financial viability of the application

It measures the quality of the model's outputs

When evaluating whether a foundation model meets business goals, which of the following is a key consideration?

The model's ability to improve productivity and user engagement

A medical research team has built a RAG system that answers questions from clinical trial documents. They need to ensure that answers are not only relevant but also faithful to the source documents, because incorrect medical information could be harmful. They have a limited budget for human review. Which evaluation strategy should they use?

Use human review on all answers

Use only retrieval precision metrics

Use ROUGE and BLEU to measure overlap with reference answers

Use automated faithfulness metrics like BERTScore on a large sample, and human review only on cases where the metric score is low

AWSCertified AI Practitioner

Domain 3Objective 4

Task Statement 3.4: Describe Methods to Evaluate FM Performance. AIF-C01 Practice Questions (Page 3)

Part of the Content Domain 3: Applications of Foundation Models domain, which makes up ~32% of our current practice bank. AWS does not publish an official question count, but from its 90-minute exam (~35–60 total, ~11–19 in this domain), expect 3–5 from this objective — we provide 40 practice questions to prepare you well beyond it. (estimate)

40questions here

8free pages

14concepts

Questions 11–15

11foundation · easy
When evaluating an agent that uses tools, what does 'tool use correctness' assess?
Select an answer first
12foundation · easy
Which metric is commonly used to evaluate the performance of an AI agent?
Select an answer first
13foundation · easy
Why is cost per interaction an important metric for business evaluation of AI applications?
Select an answer first
14foundation · easy
When evaluating whether a foundation model meets business goals, which of the following is a key consideration?
Select an answer first
15expert · hard
A medical research team has built a RAG system that answers questions from clinical trial documents. They need to ensure that answers are not only relevant but also faithful to the source documents, because incorrect medical information could be harmful. They have a limited budget for human review. Which evaluation strategy should they use?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to AIF-C01

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by AWS. “AIF-C01” is a trademark of its owner, used for identification only.

Task Statement 3.4: Describe Methods to Evaluate FM Performance. AIF-C01 Practice Questions (Page 3)

Questions 11–15

When evaluating an agent that uses tools, what does 'tool use correctness' assess?

Which metric is commonly used to evaluate the performance of an AI agent?

Why is cost per interaction an important metric for business evaluation of AI applications?

When evaluating whether a foundation model meets business goals, which of the following is a key consideration?

Keep studying — Content Domain 3: Applications of Foundation Models