Prompt engineering: how to get the answer you need from a model

Why this is worth knowing

• To understand how the share of correct answers rises without changing the model and without new budget
• To know what to ask a vendor: which sample the measurement ran on, and how the instruction gets revised from here
• To tell work on wording apart from the expensive options — fine-tuning and switching providers

What is prompt engineering in plain words

A new manager is handed an instruction for sorting incoming requests. The first version says "process the enquiry" — and half the requests land in the wrong department. The instruction is rewritten: the departments are listed, three examples of borderline cases are added, along with a rule that says "if none of them fits, send it to the general queue."

The errors disappear not because the manager got smarter. The wording of the task changed.

Prompt engineering is the same thing, but with a model. A prompt is the text of the task sent to the model on every request. Prompt engineering is the work around it: collect examples, measure the share of correct answers, revise the wording, measure again.

There is one difference from working with a person, and it is decisive. Testing a new version of the instruction on a hundred cases takes minutes, not a quarter.

How it works

Then there is the question of what exactly gets revised. Experience with reviewing instructions narrows the changes down to four types: a strict answer format instead of free text, two or three worked examples written into the text itself, an explicit rule for the "nothing matched" case, and a ban on inventing missing fields. The last one removes part of the hallucination problem — answers where the model fills a gap with plausible fiction.

The fixed part of the instruction is usually moved into a system prompt. A model remembers nothing between requests, so the full context is sent afresh on every call. Caching helps here: an unchanged opening section of the request is read from cache and billed at a reduced rate. The size of the discount depends on the provider and should be checked against its price list — hence a second lever: the boilerplate section is kept stable so that the cache is hit more often.

An example from practice

A company sorts incoming email into seven types of enquiry. The flow is around 3,000 letters a month. The first wording describes each type in a single line and asks the model to "determine the category."

The figures below are calculated benchmarks for a process of this profile, not a report on a specific deployment. The measurement method is stated so that it can be repeated on any company's own data.

The measurement: 100 letters labelled by hand by two employees, with borderline cases reviewed separately. The share of correct categories on the first wording is around 70%. An error review shows that almost all of them fall on two adjacent types and on letters containing two enquiries at once.

Then come three revisions, one at a time. A strict answer format with a fixed list of values. Three examples of those very adjacent cases written into the instruction text. A rule: if there is more than one type, return the primary one plus a "human review required" flag.

The model was never changed. What changed was the text of the task and the order of testing. The remaining 9% go to a human via the flag — and that is built into the process in advance, rather than discovered through complaints.

What this gives a business

• Quality improves without new budget. Revising the wording costs analyst hours; changing the model means moving the whole system and re-running acceptance. The order of moves runs from cheap to expensive: wording, then retrieval over an in-house knowledge base, then fine-tuning.
• A number appears that can be agreed on. "Share of correct classifications on a sample of 100 cases" is an acceptance criterion for a contract. Without it, acceptance comes down to "the answers look fine, more or less."
• Errors become manageable. The rule "if unsure, flag it and hand it to a human" turns part of the errors into an honest handover. From there it is a decision about what share of manual reviews is acceptable in money terms.
• The bill for requests is predictable. The instruction goes to the model on every request and is counted in tokens. Trimming the boilerplate section shows up in the invoice immediately — the rough order of magnitude can be estimated with the AI budget calculator.

When this can be skipped

One-off tasks need no refined instruction. Writing an email, shortening a text, sketching out headline options — an ordinary request is enough here, and a person sees the result immediately and edits it by hand.

Separate work is also unnecessary where a model is already embedded in a finished product: the instructions inside were written by the vendor, and only settings are exposed from the outside. The point appears at volume — when the same task repeats hundreds of times a month and the answer travels onward without a human looking at it.

There is a boundary on the other side too. If the errors come from missing data — the model simply does not know the company's contracts — wording will not save the day. That is treated by feeding the relevant documents into the request, and that is already context engineering.

What is worth checking

First: changing the model wipes out part of the tuning. Wording refined for one model produces a different error rate on another — sometimes higher, sometimes lower. That is why the sample and the measurement are kept: during a migration the sample is run again and compared against the previous figure.

Second: what should be requested from a vendor is the sample itself and the measurement protocol, not the final percentage. A percentage calculated on convenient examples says nothing about the real flow. What matters is whether rare and borderline cases made it into the sample.

Third: the instruction lives alongside the process. A new type of request appears, the rules change — the wording is revised and re-measured. Without that, the share of correct answers quietly slides over a few months, and it will become visible through customer complaints.