What is a context window, in practice?

The context window is all the text a model can reference when it answers — its working memory for one request. Anthropic’s documentation describes it as “all the text a language model can reference when generating a response, including the response itself.”^[1] That last clause matters: the answer the model is writing counts against the same budget as everything you sent in.

It is measured in tokens, the same unit you are billed in. In a back-and-forth conversation, the window fills up turn by turn. Each request carries the whole history so far plus the new message as input, and the model’s reply becomes part of the input for the next turn.^[1] A long conversation therefore grows the window steadily, and you pay to re-send that accumulated history every time.

Bigger windows are not automatically better. As the token count climbs, accuracy and recall tend to fall off — a pattern the docs call “context rot.”^[1] A model with room for a million tokens still does its best work when the window holds what is relevant and little else.

It helps to make “a million tokens” concrete. OpenAI’s rule of thumb for English text is “1 token is approximately 4 characters or 0.75 words”^[2] — so 100 tokens is close to 75 words. A 200,000-token window, on that ratio, holds somewhere around 150,000 words of ordinary text, several hundred pages’ worth. A model with a full million-token window could hold a window that size five times over. In practice you rarely want to fill it: beyond the context-rot problem, you are paying to send everything in the window on every single turn, cached prefixes aside, so a bigger window you don’t need is a bigger bill you don’t need either.

When a request would run past the limit, the behavior depends on the model. On newer models the request is accepted and generation simply stops if it reaches the ceiling, reporting a “model_context_window_exceeded” stop reason; older models reject the request up front instead.^[1] Either way, the fix is the same: send less, or summarize the older parts of the conversation so the window stays focused.