What is context compaction and why long sessions need it
Why this matters
- Understand why a long session with a model gets more expensive toward the end, and how to keep that in check
- Ask a vendor what happens to a task's history once the window fills up
- Budget for long conversations in advance, before launch
What is context compaction in plain words
A meeting is in its fourth hour. The transcript has grown to forty pages. A newcomer is not handed the whole thing. They get a one-page digest: why everyone gathered, what has been settled, what is left.
The same thing happens with a model. It does not remember the past on its own: the entire conversation is sent again with every turn. The context window is finite, and a long conversation runs into its edge.
Context compaction replaces that bloated history with a short summary. Old turns are removed. A digest goes at the top of the request: the goal, the decisions, the facts, the open questions. Work continues from the same point, and the input volume of a turn drops back to where it started.
How it works
The verbatim tail is there for coherence. The last few turns tell the model what was being discussed a minute ago, while the summary holds the frame of the whole task.
The summary template is the setting that matters most. Left to write freely, the model retells things differently each time, and one day the contract number falls out. The fields are defined in advance and stay unchanged between sessions. A summary like that can then be checked by eye in half a minute.
Whatever cannot be lost is pinned down separately. Payment details, document links, approval status are stored outside the conversation — in a task card, a file, or a database. The conversation gets compressed; the database record stays complete.
An example from practice
The figures below are calculated reference points for a long working session, not a report on a specific deployment. The method is simple: a page of business text at 1,800 characters yields roughly 800 tokens in Russian, and the session is counted as twenty turns of one page each.
The gap is 16,000 tokens against 4,800 — roughly threefold. The compression itself also costs money: the model reads the discarded history once, in full. Across twenty turns that request comes up once or twice, and it pays for itself by mid-session.
The savings grow with the length of the work. At five turns, compaction delivers nothing. At a hundred, it decides whether the task survives to the end.
What this gives a business
- The price of a long session stops being a surprise. Without compression every turn costs more than the last, and doubling the length of a conversation more than doubles the bill. With a summary, the volume hovers around a constant. The rough order of the sum for a given process can be sketched out with the budget calculator.
- Long tasks become feasible at all. Working through a sixty-page contract, a multi-day exchange with a client, a step-by-step incident review — none of it breaks off halfway anymore.
- A control point appears. A summary takes a person half a minute to read. It shows what the model considers settled, and an error is caught before it reaches the client.
- The work survives a restart. The summary is stored separately, so the task can be picked up the next day without retelling the whole history from scratch.
When this can be skipped
Short exchanges need no compression. One question and one answer in support fit into a thousand or two tokens; the edge of the window is nowhere near.
Compaction is equally unnecessary where every request stands alone. Knowledge base search, parsing a single document, classifying emails — no history accumulates between requests at all.
What is worth checking
First: summary quality is judged by measurement, not by eye. Take two or three dozen completed sessions and count how many of them required asking again about something already stated. That share is the only honest indicator that the field template has been put together correctly.
Second: compression consumes tokens of its own, and on short sessions it drives the bill up. The trigger threshold is tuned on real data over a couple of working days.
Third: a vendor is worth asking for a breakdown of the input volume of a single turn — how much is the instruction, how much the summary, how much the live conversation. That breakdown shows immediately where the money goes.
Frequently asked questions
What is context compaction in plain words?
It is the act of condensing an accumulated conversation into a short digest. Old turns are removed from the request and a summary is placed at the top instead: the goal of the task, the decisions made, the verified facts, the open questions. The model picks up from the same point but reads one page instead of forty. The bill per turn falls along with the volume.
How does compaction differ from simply truncating the history?
Truncation throws out old turns wholesale: the last few remain, everything earlier vanishes without a trace. Compaction first retells what was discarded and keeps that retelling in the request. Truncation is cheaper and works fine for short support tickets. Compaction is needed wherever a decision was made at the start of the conversation and has to survive to the end.
What gets lost in compression, and how can it be retained?
What gets lost is any detail the summary template did not ask for: exact wording, document numbers, caveats raised by the client. Two techniques fix this. Critical facts are duplicated outside the conversation — in a task card, a file, or a database. The summary template is defined field by field in advance rather than left to the model's discretion. Testing runs on a real sample: how often did compression force a question to be asked twice.
When should compaction be switched on?
The threshold is set by how full the context window is — usually somewhere around 70–80%. Compressing earlier does not pay off: each compression costs tokens of its own, because the model has to read the entire old conversation. Compressing later is risky: the request hits the edge of the window and breaks off. The exact threshold is found by measuring two or three days of real sessions.
What does the compression itself cost?
A single compression is one extra request: the whole discarded history goes in, and a one- to two-page summary comes out. A long session sees two or three of these. It is priced by the same formula as any other operation: input tokens plus output tokens, multiplied by the rate. Savings appear once a session runs longer than two dozen turns.
Let’s discuss your project?
Tell us about your process — we’ll suggest where AI pays off fastest.
Related articles
What is a context window, and why does the model "forget"?
A context window is the limit on how much text a model can see in a single request: instructions, documents, the question, and the conversation history all count together, and anything past the limit is dropped silently.
Long context: when it pays to feed the whole document
Long context is a model's ability to hold dozens of pages in a single request: a contract, a policy manual and an email thread are read whole, without being cut into pieces.