Analyze, Monitor, and Tune AI-powered Business Solutions AB-100 Practice Questions (Page 10)

Questions 46–49

1. 46application · medium

   A healthcare organization is deploying an AI agent that schedules patient appointments. They need to monitor the agent's performance but have strict data-privacy requirements. Which monitoring process should they design?

   • AMonitor only the infrastructure metrics (CPU, memory) and ignore application-level metrics.
   • BLog all conversation transcripts to a central analytics platform for detailed review.
   • CUse a third-party monitoring tool that stores data in a public cloud region outside the organization's country.
   • DCollect aggregated metrics such as success rate, average handling time, and user satisfaction, while redacting personal health information from logs.
   Reveal Solution Discussion Select an answer first
2. 47expert · hard

   A company has deployed an AI agent that handles customer refunds. The agent's performance metrics show a high task completion rate, but user satisfaction scores are low. The team suspects that the agent is completing tasks incorrectly or in a way that frustrates users. They need to identify the root cause. Which monitoring approach should they use?

   • AFocus only on the task completion rate and ignore user satisfaction for now.
   • BRetrain the model on all historical refund requests to improve accuracy.
   • CIncrease the confidence threshold for the agent to reduce incorrect completions.
   • DCorrelate task completion telemetry with user feedback and conversation logs to identify mismatches.
   Reveal Solution Discussion Select an answer first
4. 48foundation · easy

   When analyzing backlog data to understand AI agent usage, which metric is most directly derived from backlog items?

   • AModel training time
   • BNumber of unresolved user-reported issues
   • CNetwork latency between user and agent
   • DCPU utilization of the agent's host server
   Reveal Solution Discussion Select an answer first
5. 49application · medium

   An AI agent that handles expense report approvals is returning unexpected errors. The telemetry shows a spike in 'invalid_request' errors from the underlying model API, but the agent's logic seems unchanged. Which AI-based tool should the team use to analyze the issue?

   • AUse Azure Service Health to check for an outage in the model API.
   • BUse Azure Monitor's built-in AI anomaly detection on the error telemetry.
   • CUse Azure Log Analytics to manually filter logs for the error code.
   • DUse Azure Advisor to get recommendations on fixing the invalid_request errors.
   Reveal Solution Discussion Select an answer first