What context rot is and how to hold quality steady

Why this matters

• Understand why answers degrade as document volume grows while the tasks stay the same
• Know what keeps quality steady across long conversations and large file bundles
• Ask a vendor for a measurement at full volume, not a demo on three examples

What context rot is in plain words

A folder sits on the desk. One certificate is found in a second. Two hundred certificates sit in the same place, and finding the right one takes all evening — with some pages skimmed rather than read.

A model behaves much the same way. Hundreds of pages fit into a context window: Claude's limit of 200,000 tokens corresponds to roughly 500 pages of text. But only what fits the limit goes in, and attention inside the bundle is distributed unevenly: a detail from the middle is picked up less often than the same detail in a short request.

Hence the name. No limit is exceeded, there is no error, the answer looks confident — and the share of correct answers slides downward.

How it works

1. Context is assembled from four sources. The system instruction, conversation history, attached documents, and the results of search across the knowledge base. All four grow, and usually unnoticed.
2. Capacity and accuracy are different quantities. A stated window size says how much text will be accepted. How much of it will actually be used is a matter for a separate measurement on the company's own data.
3. Similar documents interfere more than different ones. Five contracts of the same type differing in a single clause confuse a model more readily than five documents from different departments.
4. A long conversation weighs itself down. The whole exchange is sent again with every new message; input volume grows with it and the share actually used falls. The order of magnitude for a specific task comes from the volume calculator.
5. The threshold is found by measurement. The same sample of questions is run at short and at long context, and the shares of correct answers are compared.

A subtlety about long windows: long context wins time at the start, when search across the knowledge base has not been built yet. At volume, fragment selection against the specific question wins.

An example from practice

What follows is a modelled scenario for a company of this profile, not an implementation report. Every figure comes with its method so it can be reproduced on a company's own data.

A legal department checks incoming contracts against an internal policy. The policy runs 60 pages, a contract 20–40 pages, and there are ten questions per contract. In the first version, the full policy and the whole contract go into the request.

The measurement is set up as follows. Take 150 contracts from the previous quarter with answers verified by hand by a lawyer. Count the share of clauses identified correctly — one metric, one sample, two operating modes. Mode one: the full bundle at once. Mode two: search across the policy, three to five relevant clauses, and the relevant section of the contract.

The threshold is set by the business, not by engineering: the share of missed clauses must stay at or below 3%, because the cost of a single miss in a contract runs into hundreds of thousands of roubles. If mode one fails the threshold and mode two clears it, the decision follows the table below rather than a gut feeling.

What this gives a business

• Quality becomes a verifiable quantity. A sample of tasks with known answers, two runs, two shares — that is how any change to the system is tested before work on it begins.
• Cheap levers appear. Trimming history to the last few exchanges, a summary instead of the full transcript, fragment selection instead of the full bundle — three techniques that require no change of model.
• Comparing vendors becomes honest. Comparing promised window sizes is pointless. The share of correct answers on a company's own sample at working volume compares directly.
• The bill stops growing blind. Excess text in a request is paid for on every call. Fragment selection removes part of the cost and part of the errors at once.

The mechanics of compressing history are covered in context compaction, and matching material to the question in search across an internal knowledge base.

When this can be skipped

Short tasks get by without any preparation at all. A one-page letter, a product card, a single invoice to review — small volume, few details, nothing to select.

The same goes for one-off work: a single pass over an annual report is easier to hand to the model whole and proofread by hand. Building search for one run costs more than rereading the answer.

Fragment selection pays for itself where the same process repeats dozens of times a day across a growing pile of documents.

What is worth checking

First: a vendor's demo runs on short examples, while the real process runs on long ones. Ask for the same sample to be run at realistic document volume and for both shares of correct answers to be shown.

Second: outdated editions of documents in the knowledge base cost more than missing ones. The model finds the text, answers confidently, and answers by a rule no longer in force. Policies get versioned before launch, not after the first dispute.

Third: the process needs a written volume ceiling — how many pages go into a single request. The figure comes from measurement and is revisited when the model changes. Without it, context grows on its own, and the drop in quality is noticed through complaints.