Long context: when it pays to feed the whole document

A context window is the ceiling on how much a model can hold in a single request. Inside go the instruction, the document and the question. It is measured in tokens.

Common models top out at roughly a million tokens — on the order of fifteen hundred pages. Long context is the practice of using that whole window: the document is handed over in one piece, with no cutting into fragments. How the window itself works is covered on the page about the context window.

Why does this matter

• To understand why parsing a single contract costs pennies while the same practice across an entire knowledge base produces a surprise bill
• To know when it pays to hand the model the whole document and when four pages will do
• To ask a vendor for the size of a single request in tokens, not just the window size advertised with the model

What is long context in plain words

Before a meeting, a lawyer is handed a deal folder. The contract, the annexes, the schedules of disagreement, the correspondence — all within reach. Questions are answered on the spot, without a trip to the archive for every reference.

Long context is the same folder for a model. A single request carries not a paragraph but the entire document, and the question is asked against it. The size of the folder is measured in tokens: common models top out at roughly a million, which is on the order of fifteen hundred pages.

The size of the folder is precisely the context window. Long context is a way of using it: the document is not cut up, it is handed over whole.

The figures below are calculated benchmarks for a company of this profile, not a report on a specific deployment. Each comes with its method, so it can be reproduced on any set of data.

How does it work

1. The size of the document is measured. A public benchmark for Russian text: 32 thousand tokens is roughly 60 pages, or about 530 tokens per page. A contract of that size takes up exactly those 32 thousand.
2. A decision is made about what goes into the request. The whole document, a subset of sections, or retrieved fragments. That decision is what determines the bill.
3. The document is sent together with the question. The model sees text only inside the request. It does not remember the previous file: on the next question it is sent again.
4. The fixed part is cached. The same policy manual goes to the model a hundred times a day. Repeat submissions are billed by the provider at a reduced rate.
5. Time and accuracy are measured. Volume drags response latency and request cost along with it. Answers drawn from the middle of the document are checked with a separate sample.

The main trap is simple. The document goes into the request afresh with every question. Twelve questions about one contract means twelve times 32 thousand tokens.

A case from practice

A procurement department reviews supply contracts: 40 documents a month, 60 pages each on average. Each contract draws about 12 questions — deadlines, penalties, termination terms.

The full contract runs to roughly 32 thousand tokens by the public benchmark. Twelve questions come to 384 thousand input tokens per document, about 15 million across the monthly flow. The arithmetic here is plain: volume multiplied by the number of questions and by the flow — no measurement involved.

The volume is then split across two modes. The first pass sends the whole contract in one request to build a map of the sections. The remaining eleven questions are aimed at four or five retrieved pages, about 3 thousand tokens per question. Monthly volume falls to roughly 2.6 million tokens.

Accuracy is checked throughout on a shared sample: 50 questions with answers labelled by a lawyer by hand. The run is done before and after the switch of modes.

What does this give a business

• Documents are parsed with no preparation. A contract, a policy manual, a technical brief go to the model exactly as they are. No labelling, no knowledge base, no integration: the first result arrives on the day of launch.
• Links between sections are visible as a whole. A clause on penalties contradicts an annex on page 47. That kind of thing surfaces only when both pages sit in the same request.
• The bill becomes calculable. Volume in tokens is multiplied by the rate and by the number of questions before anything starts. The order of magnitude for a given process can be sketched with the budget calculator.
• A choice of mode appears. Where the questions are repetitive, part of the volume shifts to fragment search. The bill covers pages instead of the whole volume.

When can this be skipped

Short enquiries do not call for long context. A customer question and an order record take a couple of thousand tokens. A million-token window sits idle here. Idling costs nothing: billing follows the volume actually sent.

Nor is it needed where the answer always sits in one known section. A three-page instruction is sent whole and costs little. The choice of mode starts to move money at around tens of pages per request.

What is worth checking

First: accuracy in the middle of the volume. Answers to questions from the start and the end of a document are usually steady; the dip falls in the middle. A sample of 30–50 labelled questions shows exactly where the working boundary runs for a particular set of documents.

Second: response time. Fifty pages in a request means seconds of waiting. In customer support that shows far more than in an overnight contract review. The measurement is taken on real traffic, not on a single test file.

Third: the breakdown of a single request. Ask the vendor to show how many tokens go to the instruction, how many to the document itself, how many to the message history. That reveals what to move to cache and what to move to fragment search — and how much request preparation will change the bill.