Lesson 6: Context
In this lesson, we will break down one of the key concepts when working with large language models — the context window. Understanding how context works will help you build more effective dialogues with neural networks, avoid losing important information, and consciously approach the management of long conversations.
What is a context window?
The context window is the maximum amount of information (measured in tokens) that a model can simultaneously hold in its memory while generating a response. Simply put, it is the model's workspace within a single dialogue.
Imagine you are talking to a person with limited short-term memory. They only remember the last few minutes of the conversation. Everything said earlier is forgotten. This is roughly how the context window works in language models.
What goes into the context window?
- Your current prompt
- The entire previous chat history (your questions and the model's answers)
- System instructions and settings
- Uploaded documents or files
Tokens: the unit of context measurement
To understand the size of the context window, you need to understand the concept of a token. A token is the smallest unit of text that a model processes. It can be:
- A whole word (for example, "cat");
- A part of a word (for example, "neuro" in the word "neural network");
- A single character or punctuation mark.
For English, there is a general rule of thumb: 100 tokens ≈ 75 words. It is important to understand that this ratio is different for Russian. Due to the specifics of tokenization, Russian text takes up about 2-2.5 times more tokens than English text of the same semantic volume. For example, on average, one Russian word accounts for ~3.3 tokens.
If you pay per token, it is more cost-effective to write prompts in English rather than in Russian!
Context window sizes of modern models
Models are constantly evolving, and their context windows have grown significantly. Here is the current data for popular models:
| Model | Context window (tokens) | Approximate text volume (in English) | Approximate text volume (in Russian) |
|---|---|---|---|
| OpenAI ChatGPT 5 | 400,000 tokens | ~300,000 words / ~600 pages | ~120,000 words / ~240 pages |
| Anthropic Claude Opus 4.8 | 1,000,000 tokens | ~750,000 words / ~1,500 pages | ~300,000 words / ~600 pages |
| Google Gemini 3.1 Pro | 1,000,000 tokens | ~750,000 words / ~1,500 pages | ~300,000 words / ~600 pages |
For comparison: the first ChatGPT at launch had a context window of only 8,192 tokens — about 15 pages of text. Modern models are capable of holding a volume comparable to several books in their memory.
Important: The context window includes both the input text and the output text (the model's response). That is, if you upload a document of 100,000 tokens, the model only has 28,000 tokens left to respond within a 128,000 limit.
How do models remember the dialogue?
An important nuance: language models, by their architecture, do not have built-in memory. Each request is processed independently. So how does a dialogue work?
When you communicate with ChatGPT or another neural network through an interface, the system automatically inserts your entire chat history into each new request. As a result, with each new message, your invisible prompt gets longer and longer.
How do models handle context overflow?
When the volume of the dialogue exceeds the context window limit, models behave differently. This is critically important to understand for managing the conversation.
OpenAI (ChatGPT): OpenAI models do not allow you to reset the context automatically. When the limit is exceeded, the oldest messages are simply discarded (truncated). At the same time, the history in the chat interface may remain visually intact, but the model actually does not see the beginning of the dialogue. This can lead to the neural network forgetting important instructions or input data provided at the beginning of the conversation, and it will start responding illogically.
Other models:
- Claude (Anthropic) stores the entire history until the limit is reached, after which truncation also occurs.
- Gemini uses an approach similar to ChatGPT, prioritizing the latest messages.
Important: Sometimes the "context limit exceeded" error message is confused with a rate limit. These are different things. Exceeding the context is solved by shortening the dialogue, not by waiting.
Context reset function
Working with long dialogues in OpenAI can be inconvenient precisely because old messages are truncated unnoticed by the user, and there is no special tool to reset the context in the standard interface.
In some services, this problem is solved. For example, BotHub. A special context reset button is implemented here. It allows you to manually clear the model's memory in the current dialogue without starting a new chat.
Practical tips for managing context
So, how do you effectively work with the context window so as not to lose important information?
Start a new dialogue for a new task. If you are switching to a completely different topic or a new stage of a project, it is better to open a new chat. This will give the model a clean slate and the full volume of the context window for the new task.
Periodically summarize key information. In long conversations, it is useful to refresh the model's memory:
- "Let's summarize: our main goal is ..., we have already figured out that ..., now let's move on to ..."
- "As a reminder, at the beginning of the conversation we agreed that the target audience is ..." This brings important points back into the active part of the context.
Break large tasks into subtasks. Instead of one giant dialogue dedicated to, for example, writing an entire thesis chapter, break the work into parts: first the structure, then individual sections in new chats.
Monitor the context fill percentage. Some interfaces display how much space in the context window is already occupied. This will tell you when you should think about resetting or shortening the dialogue.
The context window is the working memory of a language model. Understanding how it works and its limitations is a key skill for effective work. Modern models can process hundreds of pages of text, but their performance may decrease as the window fills up.
Remember that old messages can disappear from the model's memory unnoticed by you. Use context management strategies.