Manage the Testing of AI-powered Business Solutions AB-100 Practice Questions (Page 7)

Questions 31–35

1. 31application · medium

   An organization has deployed an AI agent that answers employee HR questions. The agent's responses are often accurate but sometimes the agent fails to understand the user's intent, leading to irrelevant answers. Which metric should the team prioritize to evaluate the agent's effectiveness?

   • AUser satisfaction score
   • BResponse generation time
   • CIntent recognition accuracy
   • DNumber of conversations handled
   Reveal Solution Discussion Select an answer first
2. 32foundation · easy

   Which activity is a key part of the testing process for an AI agent to ensure it performs reliably in production?

   • ADeleting all test data after the initial testing phase
   • BMonitoring the agent's performance and behavior after deployment
   • CAvoiding user feedback to prevent bias in the testing process
   • DPerforming testing only once before deployment
   Reveal Solution Discussion Select an answer first
4. 33expert · hard

   A testing team is using Copilot to create test cases for an AI-powered HR onboarding solution. The solution includes multiple user roles and approval workflows. The team wants to ensure thorough testing coverage while managing the effort. Which strategy should they adopt?

   • AUse Copilot to generate a baseline set of test cases from the requirements, then manually add edge cases and role-specific scenarios.
   • BAsk Copilot to generate all possible test cases automatically without review.
   • CManually create all test cases to ensure accuracy and control.
   • DUse Copilot only to generate test data, not test cases.
   Reveal Solution Discussion Select an answer first
5. 34application · medium

   A company is deploying an AI agent that handles customer refund requests in Dynamics 365 Customer Service. The agent must be tested before production rollout. The testing team has limited time and wants to ensure the agent performs reliably across different customer scenarios. Which approach should the team take first?

   • ADeploy the agent to production and monitor user feedback to identify issues in real-time.
   • BAsk the development team to review the code and manually test the agent's responses for a few sample inputs.
   • CCreate a structured test plan that includes unit tests for each component, integration tests for the agent's interactions with Dynamics 365, and end-to-end tests covering common refund scenarios.
   • DRun only end-to-end tests on the most common refund scenarios to save time and quickly identify major issues.
   Reveal Solution Discussion Select an answer first
6. 35foundation · easy

   Which metric is most appropriate for evaluating how often an AI agent provides a correct and useful response to user queries?

   • ALatency
   • BAccuracy
   • CAvailability
   • DThroughput
   Reveal Solution Discussion Select an answer first