A company has an AI agent that generates marketing copy. They want to evaluate the creativity and brand alignment of the outputs. They have a test set of 50 prompts. Which evaluation method is most suitable?

Manual evaluation by marketing experts using a scoring rubric.

LLM-as-a-judge evaluation with a prompt for creativity.

Automated evaluation using a grammar checker.

A company has an AI agent that provides medical information. They need to evaluate the agent for factual accuracy and safety. They have a test set of 500 queries with verified answers. The team includes medical experts but they have limited availability. Which evaluation method is most appropriate?

Automated evaluation using LLM-as-a-judge for both accuracy and safety.

Automated evaluation for factual accuracy, followed by expert review of only the responses flagged as unsafe or uncertain.

Manual evaluation of all 500 queries by medical experts.

A/B testing with real patients.

An AI agent for a bank was evaluated, and the results show a 95% accuracy rate. However, a deeper analysis reveals that the agent incorrectly approves high-risk loan applications. What should you do?

Accept the accuracy rate as sufficient.

Review the test set to ensure it includes enough high-risk cases and analyze the agent's decision patterns.

Retrain the agent on the entire test set.

Lower the agent's confidence threshold to reduce false approvals.

When reviewing the results of an agent performance test, what does a high precision with low recall indicate?

The agent returns few incorrect answers but misses many correct ones

The agent's responses are consistently fast

The agent returns many incorrect answers but captures most correct ones

The agent is both accurate and comprehensive

What is the primary purpose of reviewing the confusion matrix when analyzing an AI agent's test results?

To measure the agent's response time under different loads

To identify specific types of errors the agent makes, such as false positives and false negatives

To determine the agent's overall user satisfaction score

To visualize the agent's conversation flow

Microsoft Certified:AI Agent Builder Associate

Domain 3Objective 1

Evaluate Agent Performance AB-620 Practice Questions (Page 5)

Part of the Test and manage agents domain, which accounts for 20–25% of the AB-620 exam. Microsoft does not publish an official question count, but from its 120-minute exam (~50–80 total, ~10–20 in this domain), expect 5–10 from this objective — we provide 38 practice questions to prepare you well beyond it. (estimate)

38questions here

8free pages

3concepts

20–25%of the exam

Questions 21–25

21application · medium
A company has an AI agent that generates marketing copy. They want to evaluate the creativity and brand alignment of the outputs. They have a test set of 50 prompts. Which evaluation method is most suitable?
Select an answer first
22expert · hard
A company has an AI agent that provides medical information. They need to evaluate the agent for factual accuracy and safety. They have a test set of 500 queries with verified answers. The team includes medical experts but they have limited availability. Which evaluation method is most appropriate?
Select an answer first
23application · medium
An AI agent for a bank was evaluated, and the results show a 95% accuracy rate. However, a deeper analysis reveals that the agent incorrectly approves high-risk loan applications. What should you do?
Select an answer first
24foundation · easy
When reviewing the results of an agent performance test, what does a high precision with low recall indicate?
Select an answer first
25foundation · easy
What is the primary purpose of reviewing the confusion matrix when analyzing an AI agent's test results?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to AB-620

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by Microsoft. “AB-620” is a trademark of its owner, used for identification only.