Analyze, Monitor, and Tune AI-powered Business Solutions AB-100 Practice Questions (Page 4)

Questions 16–20

1. 16application · medium

   A company's customer-support AI agent shows a rising 'escalation rate' (percentage of conversations transferred to human agents) over the past two weeks. Telemetry shows the agent's average confidence score has dropped from 0.92 to 0.78, and user feedback mentions 'the agent doesn't understand my question' more frequently. The team must decide the next step. What should they do first?

   • AImmediately increase the confidence threshold so the agent escalates more often to avoid incorrect answers.
   • BDisable the agent and route all conversations to human agents until the issue is resolved.
   • CReview the telemetry and conversation logs to identify the specific intents or topics where confidence has dropped.
   • DRetrain the model with the latest conversation logs and user feedback to improve its understanding.
   Reveal Solution Discussion Select an answer first
2. 17foundation · easy

   When designing a monitoring process for AI agents in a business environment, which step should be included to ensure that monitoring remains relevant over time?

   • ARely solely on automated alerts without human review
   • BDefine key performance indicators (KPIs) aligned with business goals
   • CMonitor all telemetry data without filtering
   • DSet up monitoring once and review only when issues are reported
   Reveal Solution Discussion Select an answer first
4. 18foundation · easy

   What is the primary purpose of interpreting telemetry data from an AI-powered solution?

   • ATo replace the need for user feedback
   • BTo automatically rewrite the AI model's code
   • CTo increase the number of users
   • DTo guide decisions on tuning the model or infrastructure
   Reveal Solution Discussion Select an answer first
5. 19expert · hard

   A multinational corporation's AI HR assistant is deployed in multiple regions. The team wants to monitor its performance but must comply with different data-privacy regulations in each region. They need a monitoring process that provides a global view while respecting local data-residency requirements. What should they do?

   • ADeploy regional monitoring instances that store data locally, and aggregate only anonymized metrics to a central dashboard.
   • BStore all monitoring data in a single central location for simplicity.
   • CMonitor only in regions with the strictest regulations and ignore others.
   • DUse a single monitoring tool that stores data in the company's headquarters region.
   Reveal Solution Discussion Select an answer first
6. 20application · medium

   An AI agent that books travel arrangements receives user comments. Many comments say 'the agent booked the wrong flight time' and 'it ignored my preferred airline'. The team wants to improve the agent. What should they do based on this feedback?

   • AReview the conversation logs to see where the agent misinterprets user preferences.
   • BAdd more travel options to the agent's knowledge base.
   • CIncrease the temperature to make the agent more flexible in booking choices.
   • DReduce the number of steps in the booking flow to speed up the process.
   Reveal Solution Discussion Select an answer first