What Inference Is and What Makes Up the Bill for It

A token is a piece of text roughly three-quarters of a word. The model reads a request in tokens and answers in tokens, and the bill covers how many there were. The longer the request and the answer, the more expensive the call.

Every operation is priced this way: an answer to a customer, a parsed supplier invoice, a meal recognised in a calorie tracker. The cost of one call is input tokens plus output tokens at the chosen model's rate. What follows is what that bill is made of and how it gets smaller.

Why this is worth knowing

• To understand what the monthly AI bill actually consists of
• To measure payback not as "what does rollout cost" but as "what does parsing one document cost"
• To know which cost levers exist, and to ask a vendor about them

What is inference in plain language

AI spending splits in two. Training is a model reading the internet from scratch and learning language. It runs for weeks, costs millions of dollars, and the vendor does it, not the customer. Inference is every call to a finished model in day-to-day work: an answer to a customer, one invoice parsed, one contract checked.

The analogy is straightforward. Training is the years an accountant spent qualifying for the profession. Inference is an ordinary working day spent posting documents. The salary pays for the working days, not for the degree.

Hence the key consequence for a budget: what matters is the unit cost of a single operation, multiplied by volume. "What does it cost to roll out AI" is a question about a one-off project. "What does it cost to parse one incoming invoice" is the question that decides whether any of it pays for itself within a year.

How it works

The model receives a request, breaks it into tokens — pieces roughly three-quarters of a word — and predicts the continuation step by step. Every step takes computation on a GPU. The longer the request and the longer the answer, the more steps, and the more the call costs.

That is why a bill always has two columns: input tokens, meaning what the model read, and output tokens, meaning what it wrote. Output costs more, usually several times more: it has to be generated one token at a time, whereas input is processed in one pass.

A non-obvious line of spending hides here too. Sending the model the same two-page instruction and the entire previous conversation on every single call means paying for all of it over again each time. At a volume of thousands of calls a day, that becomes the bulk of the bill.

An example from practice

A company was parsing incoming supplier invoices: roughly 4,000 documents a month. The first version sent the model the full text of the instruction, formatting examples, and the whole document — which came to about RUB 11 per invoice, around RUB 44,000 a month.

Three things changed: caching was switched on for the fixed part of the prompt, the full scan stopped being sent once the text had already been extracted from it, and simple standard invoices were routed to a lighter model, leaving the strong one for the non-standard ones. The cost fell to RUB 3 per document — on the order of RUB 12,000 a month. Parsing quality was checked against the same sample of 200 invoices before and after: the error rate did not move.

What this gives a business

• A predictable budget. Once the unit cost of an operation is known, spending follows from volume instead of guesswork.
• Clear cost levers. Caching, matching the model to the task, cleaning up the context, batch processing — all of it changes the bill without rewriting the system.
• A basis for comparing proposals. A vendor who quotes a price per operation and explains what it is made of understands the subject. One who only talks about the price of the project may well not.

When this can be ignored

At small volumes — a few dozen calls a day, say — optimising inference will not repay the time spent on it. The bill will stay within a couple of thousand roubles a month, and any saving will come to less than the cost of the work needed to achieve it.

Optimisation is best left until a pilot has proven its worth. While it is still unclear whether the solution works at all and what the real volume will be, saving on tokens is premature work. Value comes first, unit cost second.

What is worth checking

Latency is the flip side of cost. A cheap model answers faster, a strong one thinks longer, and reasoning models can spend more tokens on deliberation than on the answer itself. When an assistant is answering a customer in chat, the bill matters no more than the seconds of waiting: people leave before the reply arrives. Both figures are measured together, before launch, on real requests.