5.4 Describe How Token Usage and Context‑window Management Affect Prototyping Cost, Latency, and Output Quality AI-TECHNICAL-PRACTITIONER Practice Questions (Page 3)

Questions 11–15

1. 11application · medium

   A real-time transcription service uses an LLM to generate meeting summaries. The service sends the full transcript (about 20,000 tokens) to the model. Users complain that summaries take too long to appear. Which approach would most reduce latency while keeping summary quality acceptable?

   • AUse a model with a larger context window.
   • BSplit the transcript into sections, summarize each, then combine the section summaries.
   • CSend the entire transcript but ask for a shorter summary.
   • DIncrease the max tokens for the response.
   Reveal Solution Discussion Select an answer first
2. 12application · medium

   An application sends a 10,000-token prompt to an LLM and receives a 500-token response. The developer wants to reduce the perceived latency for users. Which change would have the most direct impact on reducing the time to first token?

   • ASwitch to a model with a larger context window.
   • BReduce the number of input tokens by trimming unnecessary system instructions.
   • CLower the temperature to 0.0.
   • DIncrease the max tokens for the response to allow longer output.
   Reveal Solution Discussion Select an answer first
4. 13foundation · easy

   Why does a longer context window tend to increase latency in AI interactions?

   • AThe API has to send more data over the network, which is always slower.
   • BThe model must wait for the user to type more text.
   • CThe model has to retrieve more data from a database.
   • DThe model must attend to a larger amount of text, requiring more computation.
   Reveal Solution Discussion Select an answer first
5. 14foundation · easy

   What happens when the combined input and output tokens exceed the context-window limit?

   • AThe model automatically expands the context window to accommodate the extra tokens.
   • BThe request fails or the output is truncated to fit within the limit.
   • CThe model processes the request but with reduced accuracy.
   • DThe extra tokens are ignored, and the model processes only the first part of the input.
   Reveal Solution Discussion Select an answer first
6. 15foundation · easy

   What is prompt optimization in the context of token management?

   • ASelecting a model with a larger context window
   • BRewriting prompts to be more concise while retaining necessary information
   • CSending multiple requests to split the prompt
   • DUsing longer prompts to provide more detail to the model
   Reveal Solution Discussion Select an answer first