Analyze, Monitor, and Tune AI-powered Business Solutions AB-100 Practice Questions (Page 6)

Questions 26–30

1. 26application · medium

   An AI agent that summarizes legal documents is producing summaries that are too short and miss key clauses. Telemetry shows that the token usage is low and the 'completeness' score is below target. Which tuning action is most appropriate?

   • ADecrease the max_tokens parameter to force more concise summaries.
   • BLower the temperature to make the model more deterministic.
   • CIncrease the temperature to encourage the model to include more details.
   • DIncrease the max_tokens parameter and adjust the prompt to require including all key clauses.
   Reveal Solution Discussion Select an answer first
2. 27application · medium

   An AI agent that generates marketing copy is producing responses that are too verbose and sometimes off-topic. Telemetry shows that the average response length is high and the relevance score is low. The team wants to tune the agent. Which adjustment should they make?

   • ARemove the max_tokens limit to allow the model to generate longer responses.
   • BLower the temperature parameter and add a max_tokens limit to the model call.
   • CIncrease the number of few-shot examples in the prompt to show longer responses.
   • DIncrease the temperature parameter to encourage more creative responses.
   Reveal Solution Discussion Select an answer first
4. 28foundation · easy

   Which metric is commonly used to assess the performance of an AI agent in a customer support scenario?

   • AServer uptime percentage
   • BNumber of developers on the team
   • CNumber of lines of code in the agent
   • DCustomer satisfaction score (CSAT)
   Reveal Solution Discussion Select an answer first
5. 29expert · hard

   An AI agent that generates financial reports is producing reports with occasional factual errors. Telemetry shows that the errors are not random; they occur when the input data contains certain types of transactions. The team wants to reduce errors without affecting performance on other inputs. Which approach should they take?

   • ARetrain the model on all financial data to improve overall accuracy.
   • BUse telemetry to identify the specific transaction types that cause errors, then add targeted few-shot examples and validation rules for those cases.
   • CIncrease the max_tokens to allow the model to generate more detailed reports.
   • DLower the temperature globally to make the model more deterministic and reduce errors.
   Reveal Solution Discussion Select an answer first
6. 30application · medium

   A university's AI admissions assistant receives feedback from prospective students. The feedback includes comments like 'the assistant is too slow' and 'it gives conflicting information about deadlines.' The team wants to improve the assistant. What should they do?

   • AIgnore the feedback because it is anecdotal and not statistically significant.
   • BIncrease the assistant's response time limit to allow more processing.
   • CImmediately rewrite the assistant's prompts to be more concise.
   • DAnalyze the feedback to identify the most common themes and correlate them with performance metrics like response time and consistency.
   Reveal Solution Discussion Select an answer first