Context headroom for a dialogue
Six numbers show how many question-and-answer pairs fit into the window before the beginning is evicted, and what eats the headroom faster than anything else.
Your numbers
System prompt, tone of voice, prohibitions, answer format — everything that is inserted into every request.
How many document chunks retrieval attaches to the customer's question.
One continuous dialogue is assumed. Folding the history into a summary and clearing the session between enquiries push this limit further out.
An ordinary request review fits entirely, and the start of the conversation stays in memory until the end. For long consultations the history is folded into a summary every twenty turns — the limit moves three times further out.
- Passages attached to one turn
- 3 780
- Occupied before the first turn
- 25 460
- Share of the window taken by the constant part
- 12,7%
- Free for the conversation
- 174 540
- Growth per turn
- 3 976
- Comfortable limit
- 34
The calculation gives an order of magnitude. The exact token count depends on the model's tokenizer and document formatting — a provider's counter run on real conversations will give a closer figure.
How the calculation works
The window splits into two parts. The first is always occupied: the instruction, the passages from the knowledge base and the reserve for the answer. The second is left for the live conversation.
Words are converted into tokens with a factor of 1.4. For Russian text this is closer to reality than counting one to one. The factor is checked like this: the text of the instruction is run through the provider's token counter and the result is divided by the number of words.
The free part is divided by the growth per turn. That growth is made up of the question-and-answer pair and the passages left in the history. The quotient is the number of turns until the beginning of the conversation moves outside the window.
What eats the headroom faster than anything else
A pair of turns of 140 words weighs around 200 tokens. Six knowledge base passages of 450 words — almost 3800. A twentyfold difference.
Hence the main lever: the switch that decides the fate of the retrieved passages. As long as the passages are removed from the history after the answer, they are paid for once per turn and do not accumulate. As soon as they stay in the conversation, the dialogue grows at the rate of a document per message.
The second lever is the number of passages. Six chunks instead of three cut the dialogue length exactly in half. Answer quality does not rise twofold in return: the needed passage is usually among the first three results.
The third lever is the reserve for the answer. With a 200K window, a 20% share takes 40,000 tokens before the first message. That is justified for detailed analyses with quotations, while 5% is enough for short reference answers.
Turn length matters least of all. It becomes noticeable when the passages are removed from the history and the conversation is the only growing part.
When the headroom is enough
The guideline is simple: the comfortable limit is 80% of the calculated number of turns. In the last few per cent of the window the model is already working with a truncated history and answers worse, although it shows no errors.
For reference scenarios the headroom is almost always enough. An enquiry closes in three to five turns, the session resets, accumulation never gets started.
Reviewing a request needs headroom of 20–30 pairs. A dialogue like that holds in full, and an agreement made in the second turn survives to the final answer.
For a long consultation with a change of topic, one window is not enough at any size. Folding helps here: every twenty turns the history is replaced by a summary of 300–400 words and the original messages are discarded. The limit moves three times further out, at the cost of details agreed at the beginning.
How to stretch the headroom without changing the model
Permanent rules are moved into the system prompt once. Duplicating the instruction in every turn is common and costs window space for nothing.
Passages are attached to a specific question and removed after the answer. Links to sources are kept — they weigh tens of tokens and hold the coherence together.
The customer record and the request status are stored outside the conversation and inserted in their current form. Otherwise the window accumulates ten versions of the same status, of which only the last is correct.
The dialogue is cut by meaning. A new enquiry from the same customer starts with a clean session and a short summary of the previous one — that is cheaper than dragging the whole feed along.
What to do with the number
The result shows the boundary beyond which the start of the conversation stops influencing the answer. After that the decision is made by scenario: in some places the boundary is pushed out with folding, in others it is accepted as is and the session is reset earlier.
The breaking point in a working bot is visible in the logs: request length in tokens grows from turn to turn and hits the ceiling. A measurement across ten dialogues over a week gives the real growth per turn — that figure is substituted into the calculation in place of the word-based estimate, producing a number for the specific build.
FAQ
Why does the bot forget the beginning of the conversation
The model only sees what fits into the context window. When the conversation outgrows the window, old messages are cut off from the start — together with the customer's original question and the agreements made in the first turns. The answer still looks confident, so the memory break is noticed by meaning rather than by an error message.
How many turns can a bot with a connected knowledge base hold
At the default values — a 200K window, a 1200-word instruction, six passages of 450 words, turns of 140 words — it comes out at around forty pairs. What decides this is not message length but the retrieved passages: they weigh roughly twenty times more than the conversation itself. This is verified with the provider's token counter on ten real dialogues.
What does "passages stay in the history" mean
After the answer, the document chunks are either deleted from the conversation or left in it as part of the context. The second option is the default in many off-the-shelf builds. It preserves the coherence of answers, but every turn starts to cost as much as a small document.
Why keep a reserve in the window for the model's answer
The answer is written into the same window as the question. If the free space is smaller than the planned answer length, generation breaks off mid-sentence. A reserve of 10% of the window covers ordinary answers; for long analyses 20% is taken.
Does a million-token window settle the question
The limit moves further out but does not disappear: with 4000 tokens of retrieved passages per turn, a million is spent in two and a half hundred messages. Accuracy on long context also drops — the model is worse at finding what is needed among the surplus. Selecting passages remains useful even with a large window.
Check it on your case?
We will look at your process and give numbers for it, not for averages.
Related tools
Updated: August 28, 2026