5.4 Describe How Token Usage and Context‑window Management Affect Prototyping Cost, Latency, and Output Quality AI-TECHNICAL-PRACTITIONER Practice Questions (Page 4)

Questions 16–20

1. 16expert · hard

   A startup is prototyping an LLM-based document Q&A system. They have a limited budget and need fast responses. The documents are large (50,000+ tokens). They are considering two approaches: (A) send the entire document to the model each time, or (B) use a retrieval system to send only relevant sections. Which approach is better for their constraints, and why?

   • AApproach A, because larger context windows are always cheaper per token.
   • BApproach A, because the model can see the whole document and give more accurate answers.
   • CApproach B, because it reduces token usage, lowering cost and latency, while maintaining answer quality through relevant context.
   • DApproach B, but only if the model has a larger context window.
   Reveal Solution Discussion Select an answer first
2. 17foundation · easy

   Which technique is commonly used to manage token usage in a long conversation?

   • ASummarizing earlier parts of the conversation to reduce the number of tokens
   • BUsing a model with a smaller context window to force shorter responses
   • CSending the entire conversation history with every request
   • DIncreasing the model's context window size
   Reveal Solution Discussion Select an answer first
4. 18application · medium

   A team is prototyping an email-drafting feature. They send the user's entire email thread (often 5,000+ tokens) plus a 500-token instruction prompt to the LLM. The prototype is costing more than expected. Which change would most effectively reduce cost per request?

   • ATrim the email thread to only the most recent messages and include a brief summary of older ones.
   • BIncrease the max output tokens to allow longer drafts.
   • CSet the temperature to 0.0 for more deterministic output.
   • DUse a model with a higher token limit.
   Reveal Solution Discussion Select an answer first
5. 19foundation · easy

   How does higher token usage affect the cost of prototyping with an LLM API?

   • AIt has no effect on cost; cost is based on the number of requests.
   • BIt decreases the cost because larger requests are more efficient.
   • CIt only affects cost if the request exceeds the context window.
   • DIt increases the cost because APIs typically charge per token.
   Reveal Solution Discussion Select an answer first
6. 20expert · hard

   A team is prototyping a medical-document summarizer. They must keep costs low but cannot compromise on the accuracy of the summary. The documents are long (20,000+ tokens), and the model's context window is 8,000 tokens. Which approach best meets their needs?

   • ASplit the document into sections, summarize each section, then combine the section summaries.
   • BUse a model with a larger context window.
   • CSend the entire document and rely on the model's ability to summarize.
   • DTruncate the document to the first 5,000 tokens.
   Reveal Solution Discussion Select an answer first