Analyze, Monitor, and Tune AI-powered Business Solutions AB-100 Practice Questions (Page 5)

Questions 21–25

1. 21expert · hard

   An AI agent that generates job descriptions is receiving feedback that the descriptions are 'too generic' and 'don't reflect company culture'. Telemetry shows that the agent's responses are consistent but the 'specificity' score is low. The team wants to improve the agent without making it too verbose. What should they do?

   • AAdd few-shot examples that include specific company culture details and adjust the prompt to require those details.
   • BLower the temperature to make the model more deterministic.
   • CIncrease the max_tokens to allow longer descriptions.
   • DIncrease the temperature to encourage more creative descriptions.
   Reveal Solution Discussion Select an answer first
2. 22application · medium

   A logistics company has deployed an AI agent that tracks shipments and answers customer queries. They want to monitor the agent's performance in real time and set up alerts for anomalies. Which tool should they recommend?

   • AAzure Logic Apps
   • BMicrosoft Purview
   • CAzure Monitor with Application Insights
   • DAzure DevOps Pipelines
   Reveal Solution Discussion Select an answer first
4. 23application · medium

   A company has deployed an AI agent that automates invoice processing. The agent runs on Azure and uses Azure OpenAI, Azure AI Search, and Azure Logic Apps. The operations team wants to monitor the agent's health and performance without building custom dashboards. Which approach should they take?

   • AUse Azure Log Analytics queries to manually query logs from all services and create a custom dashboard in Azure Monitor.
   • BUse Azure Service Health to monitor the agent's runtime status and alert on any service incidents.
   • CEnable Application Insights for the agent and use the built-in Azure Monitor workbook templates for AI agents.
   • DUse Azure Advisor to get recommendations on agent performance and automatically apply tuning changes.
   Reveal Solution Discussion Select an answer first
5. 24application · medium

   A support team uses an AI agent to answer customer queries. Users can rate responses with thumbs up/down and leave comments. The team notices that many downvotes are accompanied by comments like 'the agent asked for my account number twice' and 'it repeated the same question'. Which tuning action should the team prioritize?

   • AAdd more examples to the few-shot prompt that show the agent asking for account numbers.
   • BIncrease the temperature setting of the underlying model to generate more diverse responses.
   • CAdjust the agent's conversation flow to avoid redundant information gathering.
   • DReduce the number of turns the agent can have in a conversation.
   Reveal Solution Discussion Select an answer first
6. 25expert · hard

   An online retailer's AI recommendation engine shows a steady decline in conversion rate over a month. Telemetry reveals that user engagement with recommended items has dropped, but the model's offline accuracy metrics (e.g., AUC) remain stable. The team is considering several actions. What should they do?

   • AInvestigate the telemetry for changes in user behavior or item popularity that may not be captured by offline metrics.
   • BIncrease the number of recommendations shown to users.
   • CImmediately retrain the model with the latest data.
   • DAssume the model is fine because offline metrics are stable.
   Reveal Solution Discussion Select an answer first