Manage the Testing of AI-powered Business Solutions AB-100 Practice Questions (Page 8)

Questions 36–40

1. 36expert · hard

   A team is testing an AI agent that uses a custom model to recommend products in Dynamics 365 Commerce. The model's accuracy is high, but the agent sometimes recommends out-of-stock items. The team needs to improve the agent's reliability. What should they do?

   • ARemove the custom model and use a rule-based recommendation system.
   • BAdd a validation step in the agent's workflow to check inventory before making a recommendation.
   • CAccept the issue as a limitation of AI and inform users.
   • DRetrain the model with more product data to improve its accuracy.
   Reveal Solution Discussion Select an answer first
2. 37expert · hard

   A company is deploying an AI agent that uses a custom model to approve loan applications. The agent must be tested for reliability and performance. The team has limited time and budget. They need to ensure the agent is safe to deploy. Which testing strategy should they adopt?

   • ATest the agent's performance under high load first, then address accuracy later.
   • BPrioritize testing the custom model's accuracy and fairness on a representative dataset, then run end-to-end tests on critical paths.
   • CDeploy the agent in a limited pilot and rely on user feedback to identify issues.
   • DRun only end-to-end tests on the most common loan scenarios to save time.
   Reveal Solution Discussion Select an answer first
4. 38expert · hard

   An AI agent in Dynamics 365 Sales assists sales representatives by suggesting next best actions. The agent's suggestions are often accurate but sometimes irrelevant to the current context. The team wants to evaluate the agent's effectiveness. Which metric should they prioritize?

   • ANumber of suggestions generated per conversation.
   • BContextual relevance, measured by the proportion of suggestions that are relevant to the current sales context.
   • CResponse time, to ensure suggestions are delivered quickly.
   • DUser adoption rate of the agent.
   Reveal Solution Discussion Select an answer first
5. 39expert · hard

   A team is using Copilot to generate test cases for an AI agent that handles loan applications in Dynamics 365 Finance. The agent must verify income, check credit score, and approve or reject applications. The team wants to ensure the test cases cover regulatory compliance. What should they do?

   • AProvide Copilot with the specific regulatory requirements and ask it to generate test cases that verify compliance.
   • BUse Copilot to generate test cases only for the happy path, since compliance is tested separately.
   • CAsk Copilot to generate test cases from the production data to ensure real-world coverage.
   • DRely on Copilot's general knowledge of financial regulations to generate test cases.
   Reveal Solution Discussion Select an answer first
6. 40foundation · easy

   Which element should be included in a comprehensive end-to-end test scenario for an AI solution integrating Dynamics 365 applications?

   • ATesting only the AI model's training process
   • BTesting only the performance of a single application
   • CVerification of data consistency across all integrated applications
   • DTesting only the login functionality of each application
   Reveal Solution Discussion Select an answer first