An AI agent in Dynamics 365 Customer Service handles refund requests. The agent's accuracy is high, but customers are complaining about long resolution times. The team needs to evaluate the agent's effectiveness. Which combination of metrics should they use?

Track both accuracy and average handling time to balance quality and efficiency.

Track user satisfaction scores only, as they reflect overall customer experience.

Track only average handling time, since customers are complaining about time.

Track only accuracy, since it is the most important metric.

An organization is testing an AI agent that automates invoice processing in Dynamics 365 Finance. The agent extracts data from invoices and posts them to the general ledger. The testing team wants to ensure the agent performs reliably under high volume. Which testing activity should they include?

Performance testing with a large dataset of invoices to measure throughput and error rates.

Security testing to ensure data privacy.

User acceptance testing with a few sample invoices.

Unit testing of the data extraction component only.

A team has built a custom AI model to predict customer churn. They need to validate the model's performance before deployment. The model's accuracy is high, but the team is concerned about false positives that could lead to unnecessary retention offers. Which validation criterion should they emphasize?

Recall, because it measures the proportion of actual churners correctly identified.

Precision, because it measures the proportion of predicted churners that actually churn.

Accuracy, because it measures overall correctness.

F1 score, because it balances precision and recall.

A team is developing prompts for a Copilot that generates legal summaries. The prompts must produce accurate and compliant summaries. The team has received feedback that some summaries contain outdated legal references. Which prompt best practice should they apply?

Instruct Copilot to be creative and include any relevant legal precedents.

Include the current legal source documents in the prompt and instruct Copilot to base the summary solely on those documents.

Ask Copilot to use its general knowledge to ensure the summary is comprehensive.

Use a short prompt to avoid confusing Copilot with too much information.

Which criterion is essential for validating the accuracy of a custom AI model?

Speed of training on the training dataset

Performance on the training dataset

Size of the model in terms of parameters

Performance on a held-out test dataset

Microsoft Certified:Agentic AI Business Solutions Architect

Domain 3Objective 2

Manage the Testing of AI-powered Business Solutions AB-100 Practice Questions (Page 1)

Part of the Deploy AI-powered business solutions domain, which accounts for 40–45% of the AB-100 exam. Microsoft does not publish an official question count, but from its 100-minute exam (~40–65 total, ~16–29 in this domain), expect 4–7 from this objective — we provide 44 practice questions to prepare you well beyond it. (estimate)

44questions here

9free pages

6concepts

40–45%of the exam

Questions 1–5

1expert · hard
An AI agent in Dynamics 365 Customer Service handles refund requests. The agent's accuracy is high, but customers are complaining about long resolution times. The team needs to evaluate the agent's effectiveness. Which combination of metrics should they use?
Select an answer first
2application · medium
An organization is testing an AI agent that automates invoice processing in Dynamics 365 Finance. The agent extracts data from invoices and posts them to the general ledger. The testing team wants to ensure the agent performs reliably under high volume. Which testing activity should they include?
Select an answer first
3application · medium
A team has built a custom AI model to predict customer churn. They need to validate the model's performance before deployment. The model's accuracy is high, but the team is concerned about false positives that could lead to unnecessary retention offers. Which validation criterion should they emphasize?
Select an answer first
4expert · hard
A team is developing prompts for a Copilot that generates legal summaries. The prompts must produce accurate and compliant summaries. The team has received feedback that some summaries contain outdated legal references. Which prompt best practice should they apply?
Select an answer first
5foundation · easy
Which criterion is essential for validating the accuracy of a custom AI model?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to AB-100

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by Microsoft. “AB-100” is a trademark of its owner, used for identification only.