Manage the Testing of AI-powered Business Solutions AB-100 Practice Questions (Page 9)

Questions 41–44

1. 41application · medium

   A company is deploying a customer-support AI agent that uses a custom model and integrates with Dynamics 365 Customer Service and Dynamics 365 Sales. The project team wants to establish a testing process that ensures the agent performs reliably before production rollout. They have limited time and need to prioritize testing efforts. What should they do first?

   • ADeploy the agent to production and monitor user feedback for issues.
   • BDefine measurable success criteria and key performance indicators for the agent's responses and workflows.
   • CRun a single end-to-end test with a few sample conversations to check for obvious errors.
   • DWrite detailed test cases for every possible user input before running any tests.
   Reveal Solution Discussion Select an answer first
2. 42application · medium

   A data science team has trained a custom AI model to classify support tickets by priority. Before integrating it into a Dynamics 365 Customer Service agent, they need to validate the model's accuracy and performance. Which set of criteria should they use?

   • AUser interface responsiveness, load time, and visual design.
   • BTraining loss, number of epochs, and learning rate.
   • CNumber of training samples, feature count, and model file size.
   • DPrecision, recall, F1 score, and confusion matrix on a held-out test set.
   Reveal Solution Discussion Select an answer first
4. 43application · medium

   A business analyst is crafting prompts for a Copilot that generates sales summaries. The prompts sometimes produce overly verbose and off-topic responses. Which prompt best practice should the analyst apply to improve response quality?

   • AInclude as much context as possible, even if it is unrelated to the task.
   • BProvide clear, specific instructions with desired format and length constraints.
   • CAvoid giving examples because they might bias the model.
   • DUse vague language to allow the model to be creative.
   Reveal Solution Discussion Select an answer first
5. 44application · medium

   A testing team is using Copilot to generate test cases for an AI-powered customer feedback analysis solution. They want to ensure the test cases are effective and cover various feedback types. Which strategy should they use?

   • AAsk Copilot to generate as many test cases as possible without examples to maximize coverage.
   • BProvide Copilot with examples of different feedback types and ask it to generate test cases for each type, then review and refine them.
   • CHave Copilot run the test cases automatically without human involvement.
   • DUse Copilot only to generate test data, not test cases.
   Reveal Solution Discussion Select an answer first