Context Overflow in a Dialogue
Six numbers — and it becomes clear at which turn the history stops fitting into the window, whether there is room left until the end of the shift, and how much compacting the conversation will cost.
Your numbers
A question-and-answer pair counts as one turn.
In Russian this works out to roughly 1.3 tokens per word. Excerpts from the knowledge base are included here as well, if the bot inserts them into an answer.
The instruction, tone rules, the customer card, tool descriptions — everything sent to the model on every turn.
How many such dialogues survive until the end of the shift in a single day.
The share of the working part of the window taken up by the system part and the history on the last turn of the shift. Above 100% the conversation holds together only through compaction.
By the end of the shift the history outgrows the window, and the beginning of the conversation rests on a summary. Three moves work here: a wider window, compaction on a schedule, and moving customer facts into a separate field. The line "history compaction per month" shows what the third option costs against paying extra for a window.
- The turn after which the beginning drops out
- 121
- Turns in reserve
- 0
- Context on the last turn
- 145 000
- Compactions per shift
- 1
- One compaction
- 189 RUB
- History compaction per month
- 48 000 RUB
This is an order-of-magnitude estimate. The weight of a turn depends on the language, the markup, and how many excerpts from the knowledge base the bot pulls into an answer.
How the calculation works
A dialogue grows linearly. Every question-and-answer pair lands in the history and goes to the model again on the next step. On top of the history, the system part is sent with every request: the instruction, tone rules, the customer card, tool descriptions.
First the reserve for the answer is subtracted from the model's window. Then the system part. What remains is the room for the conversation.
That room is divided by the weight of one turn. The result is the number of the turn after which the beginning of the conversation stops fitting in full. The headline number shows something else: what share of the working part of the window the dialogue occupies on the last turn of the shift.
Below one hundred percent the shift passes without losses. Above it, the history reaches the model as a summary, and the accuracy of answers starts to depend on the quality of that summary.
What moves the threshold most
The weight of a turn. It is also the most volatile value in the calculation. A reference bot with short answers stays within 300 tokens per pair. The same bot with three excerpts from a regulation runs up to fifteen hundred.
A fivefold weight moves the threshold five times closer. That is why measuring the weight against your own logs pays off sooner than any other fix.
Next comes the model's window. Moving from 32 thousand tokens to 128 pushes the threshold roughly fourfold further out: from a couple of dozen turns to a hundred and fifty. An operator's shift rarely produces more than a hundred pairs, and at this step the overflow question is usually closed.
The system part weighs less, but it is noticeable on narrow windows. An eight-thousand-token instruction with a 32-thousand window takes a quarter of the room before the first turn. With a 200-thousand window the same text is lost in the overall volume.
What the breakdown shows
The first line is the number of the turn up to which the history fits together with the system part. The second is how many pairs remain in reserve by the end of the shift. As long as the reserve is positive, the conversation survives until handover to the next operator without summaries.
The third line is useful when reconciling with the bill. The context of the last turn is the volume paid for on every request at the end of the shift. The beginning of the conversation is barely visible in the bill; the tail end is clearly visible.
The last three lines are about compaction: the number of compactions per shift, the price of one, and the monthly total across all lines over 21 working days. Zeros in these lines mean there is room to spare: there is nothing to compact while the history fits in the window.
History compaction and its price
When a conversation hits the window, it gets compacted. An almost full window is sent to the model once more, a summary of a couple of thousand tokens comes back as the answer, and the conversation continues from it. The procedure is called compaction or summarization, and it is triggered by a fill threshold or on a schedule.
The price of one compaction is the volume of context sent multiplied by the input price. With a 128-thousand-token window and one and a half rubles per thousand, that comes to about 190 RUB. The number of compactions per shift is estimated roughly: how many times the window manages to fill up after the first overflow.
The monthly total is assembled by multiplying by the number of lines and 21 working days. A dozen lines and one compaction per shift give on the order of 48 thousand rubles. This figure is compared against the surcharge for a model with a wider window — that fork is exactly why the calculation is set up.
How to measure your own values
The weight of a turn is taken with a tokenizer from an export. Select 30–50 completed dialogues from a week, count the tokens in each pair, take the median. The average lies here: two long case discussions raise it twofold while the typical dialogue stays the same.
The length of a shift is checked against the request log. The distribution is usually bimodal: some dialogues break off on the fifth turn, others stretch to a hundred. The upper edge goes into the calculation — overflow happens precisely in long conversations.
The system part is measured once, but in full: together with tool descriptions and answer examples. It grows unnoticed and by the bot's third month in operation it has often doubled.
Under what conditions the window is enough for a shift
There are three conditions. A turn stays within 600–800 tokens. The system part takes up to 5% of the window. Customer facts live in a card and are updated with a short call.
The third condition settles the question more reliably than the others. When the name, the plan and the ticket number sit in a separate field, old turns are released calmly: what was needed from them has already been carried over. Updating such a card costs pennies against a full compaction of the window.
The calculation gives an order of magnitude and the point where the memory architecture is worth a look. Exact figures come from measurements against your own logs over a couple of weeks of operation.
FAQ
At which turn does the bot start forgetting the beginning of a conversation?
The threshold is calculated like this: the reserve for the answer and the system part are subtracted from the model's window, and the remainder is divided by the weight of one question-and-answer pair. With a 128-thousand-token window, a 5-thousand-token system part and a 1000-token pair, the threshold falls on turn 121. With a 32-thousand-token window it drops to turn 25.
How does compaction differ from summarizing the history?
The mechanics are the same: an almost full window goes to the model once more, a summary of a couple of thousand tokens comes back, and the dialogue continues from it. The trigger differs. Compaction fires when a fill threshold is reached; summarization runs on a schedule or by turn count.
Why is a reserve of 2000 tokens subtracted from the window
The model needs room to finish writing an answer. History stretched across the whole window leaves the answer cut off mid-sentence. Two thousand tokens is a reserve for a detailed turn with a quotation from a document; for a reference bot with short answers, eight hundred is enough.
How much does one history compaction cost
The same as an ordinary request of that size: the context sent is multiplied by the input price. With a 128-thousand-token window and one and a half rubles per thousand, that comes to about 190 RUB. A dozen lines and one compaction per shift give on the order of 48 thousand rubles over 21 working days.
How to measure the weight of a turn on your own dialogues
Take a week's export, select 30–50 completed dialogues and run them through the tokenizer of the model in use. Calculate the median: a couple of long case discussions skew the average twofold. Look separately at the share of turns with documents pulled in — those usually account for the spread.
What gives more room: a bigger window or a shorter system part
On narrow windows the system part decides a lot. With a 32-thousand-token window, trimming the instruction from 8 to 3 thousand tokens adds about five turns out of twenty-five. With a 200-thousand-token window the same gain is lost in the overall volume, and there the weight of the conversation itself matters more.
Check it on your case?
We will look at your process and give numbers for it, not for averages.
Related tools
Updated: September 1, 2026