Context Window
A context window is the maximum amount of text, measured in tokens, that an AI model can consider at once when generating a response, including both the input and its own output.
Key takeaways
- A context window is the maximum amount of text, measured in tokens, a model can consider in one exchange.
- It includes instructions, conversation history, and the model's own response, not just the user's input.
- Modern models support context windows of hundreds of thousands of tokens, up from a few thousand in earlier models.
- A context window is not persistent memory — it resets between sessions unless a system saves and reinserts information.
- Even with a large context window, models can lose focus on details buried in the middle of very long input.
What is a context window?
A context window is the limit on how much text a language model can "see" and reason about in a single exchange. It includes everything: the system instructions, the conversation history, any documents pasted in, and the response the model generates. Once that limit is reached, older content has to be dropped or summarized for the conversation to continue.
Why context windows matter
A larger context window lets a model work with longer documents, longer conversations, or more retrieved reference material without losing track of earlier details. Early language models had context windows of a few thousand tokens; modern models routinely support context windows in the hundreds of thousands of tokens, enabling tasks like analyzing an entire codebase or a lengthy legal contract in one pass.
Context window vs memory
A context window is not the same as long-term memory. It resets between separate sessions unless a system explicitly saves and reinserts information, and everything inside it has to be reprocessed each time, which is part of why very long conversations can become slower and more expensive.
Context window limits in practice
Even with a large context window, models can lose focus on details buried in the middle of a very long input, a phenomenon often described informally as "lost in the middle." This is one reason retrieval-augmented generation remains useful even for models with huge context windows: retrieving only the most relevant passages, rather than dumping in everything, often produces more reliable answers.
Frequently asked
- What does context window mean?
- A context window is the maximum amount of text an AI model can process at once, including the conversation so far and its generated response, measured in tokens.
- What happens when a conversation exceeds the context window?
- Older parts of the conversation typically have to be dropped or summarized, since the model can no longer see everything at once.
- Does a bigger context window mean a smarter model?
- Not necessarily. A larger context window lets a model consider more information at once, but doesn't by itself improve reasoning quality, and very long inputs can still cause the model to lose focus on details buried in the middle.