5.4 Describe How Token Usage and Context‑window Management Affect Prototyping Cost, Latency, and Output Quality AI-TECHNICAL-PRACTITIONER Practice Questions (Page 1)

Questions 1–5

1. 1application · medium

   A team is building a document-analysis tool that feeds a 3,000-page manual into the LLM's context window. The model's context window is 128,000 tokens, and the manual is about 200,000 tokens. The team notices the model's answers start to ignore details from the beginning of the manual. What is the most likely cause and the best fix?

   • AThe model is biased toward recent tokens; truncate the manual to the last 100 pages.
   • BThe model's temperature is too high; lower it to 0.1 to improve focus.
   • CThe manual exceeds the context window, so early tokens are dropped; use a retrieval-augmented approach to feed only relevant sections.
   • DThe output is being truncated; increase max tokens to 10,000.
   Reveal Solution Discussion Select an answer first
2. 2foundation · easy

   In the context of large language models, what is a token?

   • AA fixed-size chunk of text that the model processes as a single unit
   • BA word, always representing one complete word
   • CA complete sentence that the model processes as one unit
   • DA single character, always representing one letter or digit
   Reveal Solution Discussion Select an answer first
4. 3application · medium

   A developer is building a multi-turn chatbot that maintains a full conversation history in the prompt. The model's context window is 8,000 tokens. After 20 turns, the conversation history is 7,500 tokens. The user asks a complex question that requires the model to reason over the entire history. What is the best approach to maintain quality?

   • AClear the conversation history and start fresh.
   • BIncrease the max tokens for the response to 2,000.
   • CSummarize the conversation history into a condensed version and include that with the latest few turns.
   • DContinue sending the full history; the model will handle it.
   Reveal Solution Discussion Select an answer first
5. 4foundation · easy

   What is a context window in an LLM?

   • AThe maximum number of tokens the model can consider at once for a single request
   • BThe memory available on the server for storing conversation history
   • CThe number of tokens the model can generate in a single response
   • DThe total number of tokens used across all requests in a session
   Reveal Solution Discussion Select an answer first
6. 5expert · hard

   A team is building a customer-support chatbot that must handle long conversations. The model's context window is 8,000 tokens. They want to maintain high-quality responses while keeping costs low. They are considering three strategies: (1) truncating old messages, (2) summarizing old messages, and (3) using a sliding window of the last 10 messages. Which strategy best balances quality and cost?

   • ATruncate old messages beyond the last 10.
   • BSend the entire conversation history every time.
   • CSummarize old messages into a condensed summary and include recent messages verbatim.
   • DUse a sliding window of the last 10 messages without any summary.
   Reveal Solution Discussion Select an answer first