What is Context Window?

A context window is the maximum amount of text — measured in tokens — that a large language model can process in a single interaction. It encompasses everything the model considers when generating a response: the system prompt, conversation history, any retrieved documents or context, and the user's current query. The context window is one of the most important practical constraints when building AI applications, as it determines how much information you can provide to the model at once.

Context window sizes have expanded dramatically since the early days of LLMs. GPT-3 offered 4,096 tokens (roughly 3,000 words). Modern models offer significantly larger windows: Claude supports up to 200,000 tokens, Gemini 1.5 Pro handles up to 1 million tokens, and some models are pushing beyond that. A larger context window means you can include more reference documents, longer conversation histories, or entire codebases in a single prompt. However, larger contexts also increase inference cost (pricing is typically per-token) and latency (more tokens take longer to process).

In practice, context windows are critical for several AI application patterns. RAG systems must fit retrieved document chunks within the context window alongside the prompt and instructions. Coding assistants benefit from large context windows that can hold entire files or project structures. Document analysis tasks like summarizing legal contracts or research papers require fitting the full document in context. Multi-turn conversation systems must manage conversation history within the context window, often using summarization or sliding window techniques when conversations exceed the limit.

An important nuance is that model performance can degrade on information placed in the middle of very long contexts — a phenomenon known as the 'lost in the middle' problem. This means that simply having a large context window does not guarantee the model will effectively use all the information provided. Best practices include placing the most important information at the beginning and end of the context, using clear structural formatting (headers, delimiters), and being strategic about what information to include rather than simply stuffing the context window to capacity.