What is Context Window?
A context window is the span of tokens that a language model can actively attend to when processing input and generating output. It represents the model's "field of view" – the amount of text it can consider simultaneously when making predictions about the next token. The context window is intrinsically linked to the attention mechanism in transformer models, which computes relationships between all tokens within the window.
The size of the context window directly impacts what the model can "remember" and reference during processing. A larger context window allows the model to maintain awareness of more distant information, capture longer-range dependencies, and work with more extensive input documents without requiring external memory systems. However, larger context windows come with computational costs, as the attention mechanism's complexity grows quadratically with the number of tokens.
In practical applications, the context window determines what information can be included in prompts to AI agents and language models. This affects system design decisions like how much conversation history to include, how many retrieved documents to incorporate in RAG systems, and whether long documents need to be split into multiple processing passes. Recent advances in model architecture have dramatically increased context window sizes, with some models supporting windows of hundreds of thousands or even millions of tokens.