Contextual chunk enrichment — what it means in plain terms

Why this is worth knowing

• Understand why knowledge base search returns similar pieces instead of the right ones
• Estimate the one-off cost of preparing the base and tell it apart from the monthly bill
• Ask a vendor for the share of correct search hits, not for overall answer quality

What contextual chunk enrichment means in plain terms

Picture a card catalogue with the pages torn out and dumped into a single pile. One page reads: "Revenue grew 3% this quarter. North America grew 12%, Europe declined 5%." Whose report, which quarter, which business unit — none of that is on the page.

A librarian handed that pile finds the right page only by luck. Asked "how did Europe do in the second quarter," the librarian pulls out a dozen similar pages. All of them are about percentages rising and falling. The right one may not be among them.

Enrichment is a line written across the top of the page before it goes into the catalogue. It reads something like: "ACME quarterly financial report, Q2 2024, section on revenue by region." From then on the page is searchable by that line as well as by its text.

In RAG, documents are cut into pieces of a few paragraphs, known as chunks. Each chunk is turned into an embedding and placed in a vector database. Enrichment inserts a note of 50–100 tokens ahead of the chunk, and only then computes the embedding.

How it works

1. The document is read in full. The model is given the entire source file plus one piece from it. The task is to describe where the piece sits in the document, not to summarise it.
2. The note stays short. 50–100 tokens: document name, period, section, who it concerns. Anything longer is pointless — the note starts to outweigh the text itself.
3. It is paid for once. Enrichment happens when the document is uploaded, not on every question. The source document is cached so it does not have to be sent again for every piece in the same folder.
4. The index is built on the enriched text. Both the vector and the exact-word search see the note. A date and a company name in the note pull the piece up on queries that previously had no match at all.
5. The model receives the original piece. What is fed into the answer is either the enriched text or the clean one. The note exists to get the piece into the results, not to be read.

The size of the context window is checked separately. Enriched pieces are longer than the originals, and fewer of them fit into the same budget. If the model handles only a short context, there is a choice to make: fewer pieces in the answer, or a shorter note.

An example from practice

A manufacturing company assembled a knowledge base from five years of policies, orders and meeting minutes. That came to roughly 4,000 documents and, after splitting, some 60,000 pieces. Employee questions sounded like this: "when was the acceptance standard changed at site two." Neither the date nor the business unit appeared inside the piece.

The order of magnitude for a base like this: about 50 cents per thousand pieces. Calculated for an inexpensive model with caching of the source document. For the whole archive that comes to roughly 30 dollars as a one-off. The ongoing top-up is calculated from the number of new documents per month and, at this volume, runs to tens of roubles.

The result is not judged by impressions. Someone assembles 50–100 real questions with the correct piece known in advance. The measure is the share of queries where the right piece landed in the top five results. The measurement is taken twice on the same sample: before enrichment and after. The gap between the two numbers is the entire effect.

What this gives a business

• Fewer "nothing found" answers. Some employee questions carry a date, a unit name or an order number. Without the note, those words never appear in the piece at all, so search cannot catch them.
• The cost is one-off and known in advance. The 50-cents-per-thousand-pieces order of magnitude is multiplied by the size of the archive, and the total is known before work begins. It does not change the monthly bill for the answers themselves — that can be estimated with the budget calculator.
• The improvement requires no change to the system. Pieces, model and interface stay as they were — only what goes into the index changes. Rolling back means reloading the base.
• A verifiable metric appears. The share of queries with the right piece in the top five is measured on one sample, before and after. The conversation with a vendor moves from "it got better" to two numbers.

When this can be skipped

Some knowledge bases are built so that pieces are self-contained without any note. Product cards with the SKU inside, policy clauses with the heading in the text, short support replies. There the gain in hits usually disappears into the measurement error.

Enrichment also delivers nothing on poor source material. A note does not repair scans recognised with errors: wrong figures stay wrong, they just get the right date on top. Recognition is fixed first, the index second.

It is equally unnecessary where the whole base fits inside the context window. A couple of hundred pages can be sent to the model with no search at all — and then neither pieces nor notes are needed.

What to check

First: the note is written by a model, and models make mistakes. On a sample of 50 pieces the notes are read by eye — to see whether the wrong date or a neighbouring section has been attached. A wrong note is worse than no note: the piece will start surfacing on the wrong questions.

Second: enrichment changes the length of a piece, and with it the volume sent to the model on every answer. Ten pieces in the prompt plus 100 tokens each is an extra thousand tokens per query. That is a monthly bill, not a one-off cost, and it is counted separately.

Third: the vendor is asked for the share of hits before and after on one sample of questions. Alongside it, the sample size and the date of the measurement. A measurement taken after enrichment without one taken before it means nothing.