What Is a Token in AI, and How Much Text Fits in a Chat?
If you use AI tools at work, you will run into the word "token" sooner or later: in a pricing page, in an error that says your input is too long, or in a note that a model has a "200k context window." Tokens sound technical, but the idea is simple, and understanding it explains several everyday frustrations, such as why a very long chat starts forgetting earlier instructions.
1. What a token is
OpenAI's help article defines tokens as the units its models use to process text. A token can be a single character, part of a word, a whole word, or punctuation, and spaces affect how text gets divided. When you send a message, the text is split into tokens, the model processes them, and it generates new tokens as its answer.
OpenAI is careful to say that a token count is not the same as a word count. The same text can produce different counts depending on the model, its encoding and the language. Even small differences matter: OpenAI notes that "red," "Red" and "red" with a leading space are not identical text and can be represented differently.
2. Rough conversions for English
OpenAI gives these rules of thumb for English text:
- 1 token is about 4 characters.
- 1 token is about three quarters of a word.
- 100 tokens are about 75 words.
Using those estimates (they are approximations, not exact counts):
| Text | Approximate size | Approximate tokens |
|---|---|---|
| A short email of 2,000 characters | about 2,000 characters | about 500 |
| A one-page memo | 1,000 words | about 1,333 |
| A 10-page report | 5,000 words | about 6,667 |
OpenAI adds that other languages can have different relationships between characters, words and tokens, so these ratios apply to English.
3. Input tokens and output tokens
Tokens flow in both directions. OpenAI separates input tokens, what you send, from output tokens, what the model writes back. Some models, called reasoning models, also produce internal reasoning tokens before the visible answer. OpenAI notes these are not shown as text but still count toward output usage, so a short visible answer can use more tokens than it appears.
This matters if your organization pays per token through an API: long pasted documents and long answers both cost more. For most people using a chat app with a subscription, tokens matter mainly because of limits.
4. The context window: the model's working memory
Every model has a limit on how many tokens it can handle at once. Anthropic's documentation calls this the context window and describes it as all the text the model can reference when generating a response, including the response itself. Anthropic compares it to a "working memory," which is different from the much larger body of text the model was trained on.
Everything in the conversation counts toward that limit. Anthropic explains that as a conversation goes on, each message and each response accumulates in the context window, and the system instructions, any attached documents, images and the model's own new output count too.
Sizes vary by model. Anthropic's documentation lists many current Claude models with a 1 million token context window and some other models with 200,000 tokens. OpenAI advises checking your model's documentation for its context window and maximum output, since these differ between models. Translated with OpenAI's three-quarters-of-a-word estimate:
| Context window | Approximate English words | Pages at an assumed 500 words per page |
|---|---|---|
| 200,000 tokens | about 150,000 | about 300 |
| 1,000,000 tokens | about 750,000 | about 1,500 |
5. Bigger is not automatically better
A huge window does not mean the model treats every word equally well. Anthropic states that as the token count grows, accuracy and recall degrade, a phenomenon it calls "context rot," and that curating what goes into the context matters as much as how much space is available.
There is also a practical reason chats forget. Anthropic notes that chat interfaces can manage the context window on a rolling "first in, first out" basis. In a very long conversation, the earliest messages, including instructions you gave at the start, may no longer be in view.
6. What this means for everyday work
- Restate key instructions in long chats. If the tool starts ignoring a rule you set early on, the rule may have scrolled out or been diluted. Repeat it, or start a fresh conversation with a short summary.
- Send only what is needed. OpenAI's suggestions for oversized input apply to everyday use too: shorten the prompt, remove repeated context, split large inputs into parts, or summarize first.
- Split very long documents. Work through a long report section by section rather than asking about all of it at once, especially when details matter.
- Do not judge size by word count alone. Tables, code, unusual names and non-English text can use more tokens than you would expect.
- Check the model's limits. If you hit an error saying the input is too long, the content plus the expected answer exceeded the window.
7. Key takeaways
- A token is a chunk of text, often part of a word; OpenAI estimates 100 tokens at about 75 English words.
- Input, output and hidden reasoning tokens all count toward usage.
- The context window is the model's working memory, and everything in the conversation fills it.
- Anthropic warns that accuracy and recall can degrade as context grows, so more is not always better.
- In long chats, restate important instructions or start fresh with a summary.