Tokens and context windows
How models measure text, and why there is a limit to how much they can read at once.
What is it?
Models do not read text letter by letter or word by word. They split it into tokens: short pieces that may be a whole word, part of a word, a punctuation mark or a space. Common English words are often one token, while rare words and many non-English scripts take more tokens.
The context window is the maximum number of tokens a model can consider at one time, including your messages, any files you attach, the instructions it was given and its own reply. When a conversation grows past that limit, older parts are dropped or summarized and the model can no longer see them.
Which models use it?
- Chat assistants such as ChatGPT, Claude and Gemini all work within a context window.
- API providers such as OpenAI, Anthropic and Google usually bill by the number of input and output tokens.
- Different models have very different context window sizes, listed in each provider's documentation.
- Providers publish tokenizer tools or token-counting features so developers can measure text.
How to use it
- Check the context window of the model you use in its official documentation.
- Start a new chat for a new task instead of continuing a very long conversation.
- Paste only the parts of a document that are relevant to your question.
- In long chats, ask the model to summarize the key decisions so far, then continue from that summary.
- Repeat critical instructions near your latest message if the conversation has become long.
- If you pay per token, keep prompts concise and ask for answers of a set length.
Benefits
- Helps you understand why long chats can "forget" earlier details.
- Lets you estimate cost and speed when using paid APIs.
- Encourages focused prompts, which usually produce better answers.
- Explains why some languages cost more tokens for the same meaning.
Limits and risks
- A large context window does not guarantee the model uses every part of it equally well.
- Details in the middle of very long inputs can be overlooked.
- Longer inputs take more time and cost more on paid services.
- Token counts differ between models, so the same text may fit in one and not another.
Prompt template
Copy it, replace the parts in [brackets] and paste it into any AI chat.
Here is a long text: [paste text]. First, summarize it in [number] bullet points that keep every fact I would need later, such as names, numbers and decisions. Then list any open questions. Keep the whole summary under [number] words so I can paste it into a new chat as context.
More in Foundations
- Large language models (LLMs)The text-predicting systems behind chat assistants: what they are and what they can do.
- Transformers and attentionThe neural network design that most modern language, image and speech models are built on.
- Training vs inferenceThe difference between teaching a model and using it, and why it matters to you.
- Open-weight vs closed modelsModels you can download and run yourself vs models you use only through a company's service.
- Choosing the right model (cost, speed, quality)How to match a model to your task instead of always using the biggest one.