Context engineering — explained simply
Why this matters
- To understand why the same model works for a peer company yet misses on in-house tasks
- To ask a vendor what the request is assembled from, not only which model sits inside
- To see a cost line that grows faster than the number of requests
What is context engineering in plain terms
A new hire on day one is given more than a task. They are given access to the contracts folder, the internal policy, the phone number of a senior colleague, the right to look through the client's history. The task is worded the same for everyone — "sort out this request" — but the outcome depends on what the person was handed.
A model works the same way. At the moment it answers, it sees exactly what was passed to it in the request: the instruction, excerpts from documents, the client record, the message history, the list of available actions. It knows nothing beyond that — neither the company's internal rules nor what was said in yesterday's conversation.
Context engineering is the rule-based assembly of that set. What to pull in, from where, in what order to place it, what to discard once the context window runs out. Work on the prompt covers how the task is phrased. Context engineering covers the material the task is solved on.
How it works
Assembly runs step by step, and each step is a separate decision that somebody makes.
A separate concern is what to do with the old material. In a long conversation the history grows with every exchange, and by the twentieth message it crowds out the very thing the request was made for. So the conversation is compressed into a short summary, and only the most recent exchanges are kept in full.
The second concern is shelf life. A rule was changed in July, yet the April version from an old export goes into the request. The model will answer confidently and incorrectly: it cannot verify how current anything is, since it has nothing beyond the text it was handed.
An example from practice
A company with 40 managers handles roughly 6,000 requests a month. An AI agent built on document search is in place: the request carries the instruction, ten retrieved fragments of internal policy and the entire conversation in full — on the order of 14,000 input tokens per request.
A breakdown of the composition showed that of the ten fragments, two or three affect the answer; the rest were selected on word overlap. The conversation was compressed into a 300-token summary, the number of fragments was cut to four, and the instruction was split into a fixed part for caching and a variable one.
Input volume now runs around 4,200 tokens. The share of requests where the answer had to be corrected was measured on a single sample of 200 handled cases before and after: it fell from 31% to 19%. The figures are illustrative, for a company of this profile, not a deployment report; the method repeats on a company's own data.
What this gives a business
- Errors become traceable. Once the composition of the request is fixed, a wrong answer is diagnosed in minutes: the export is examined and the missing document becomes visible. Without that, the wording gets edited on guesswork.
- The bill stops growing on its own. The main cost line is input volume, and it swells from conversation history and surplus fragments. The rough figure for a given process comes from the budget calculator.
- The model becomes a replaceable part. When the assembly rules are described separately from the model, switching to another one takes days rather than months. Company knowledge lives in the sources, not inside the model.
- A vendor is tested with one question. "Show what went into the model on this request" — either there is an answer, or nobody is managing the system.
When this can be skipped
A single task with a short input text gets by on an instruction. Translation, proofreading, a draft email — the model has enough in what is already written in the request; there is nowhere to retrieve from and nothing to retrieve.
Small volume likewise calls for no separate discipline. At twenty or thirty requests a week it is simpler to assemble the request by hand and watch it by eye — the time spent building an assembly mechanism costs more than the savings.
The process itself is standardised first, and only then is context assembled for it. As long as the rules for answering live in people's heads and differ from person to person, there is nothing to put into the request. Readiness is checked with a checklist in one evening.
What is worth checking
First: more context does not mean better. When the window overflows, the model holds on less well to what sits in the middle — an important rule gets lost between twenty pages of appendices. The threshold is found by measurement: sources are removed one at a time and the point where answer quality drops becomes visible.
Second: every source needs an owner and a refresh interval. A policy export made once at launch quietly turns into a source of wrong answers six months later. The date of the last update is displayed next to the source and enters the regular review.
Third: access rights travel with the data. If a manager cannot see the contracts of a neighbouring department, the model must not receive them on that manager's request either. This is settled at the context assembly step, not by filtering the finished answer.
Frequently asked questions
What is context engineering in plain terms
It is the work of shaping what the model sees at the moment it answers. The instruction, excerpts from documents, reference material, the conversation history, the available tools — all of it is assembled by rules rather than dumped into one block of text. The prompt handles how the question is phrased; context engineering handles the material the answer is built from.
How does this differ from working on the prompt
A prompt is a single instruction written by a person in advance. Context is assembled by software for each individual request: different clients pull in different documents, different roles carry different permissions. Good phrasing will not save an answer if the required document is missing from the request. The right document rescues even mediocre phrasing.
Does this require search over a knowledge base
Document search ([RAG](/en/blog/rag)) is one source of context — a common one, but not the only one. Alongside it sit the client record from the CRM, the order status, company policy, the result of a call to an external service. Context engineering answers the question of which of these enters the request now and which gets pushed out once space runs short.
How can poorly assembled context be spotted
Three signs. Answers diverge on similar requests — the composition of the context is unstable. The model cites an outdated version of a rule — the source is not being refreshed. The bill grows faster than the number of requests — surplus material is going into the request. Each sign is checked by exporting what actually reached the model.
Where does a company start on its own process
Take twenty requests that were handled manually and write down everywhere the employee looked in order to answer. That produces a list of sources. Then measure how many tokens each one takes up and cut whatever does not affect the answer. The first workable context composition usually comes in at a quarter of the original volume.
Let’s discuss your project?
Tell us about your process — we’ll suggest where AI pays off fastest.
Related articles
What is a context window, and why does the model "forget"?
A context window is the limit on how much text a model can see in a single request: instructions, documents, the question, and the conversation history all count together, and anything past the limit is dropped silently.
What is RAG: how to make a model answer from your documents
RAG is a way of working with AI where the system first finds the right passage in a company's own documents and then builds an answer from it.