AI on your own premises: when data cannot leave the building

Why this matters

• Work out whether the case at hand requires an in-house server by law, or whether it was an internal decision
• Know the price up front: graphics cards, electricity, and the person who watches over all of it
• Tell "data must not leave" apart from "data leaving feels risky" — those are two different budgets

What on-premise deployment means in plain words

Documents arrive in a foreign language. They can be carried to the translation bureau across the road and paid for by the page. Or a translator can be seated in a spare room: salary, desk, dictionaries, holiday — but not a single sheet leaves the building.

On-premise is the second option. The model is installed on company servers, an employee's query is processed inside the network, and not one line goes out. Contractors call this on-premise.

The arithmetic changes as well. In the cloud the bill grows with the number of queries: a hundred queries, a hundred small charges. Owned hardware costs the same whatever the volume. A salaried translator is paid in the month a thousand pages arrive and in the month nothing does.

How it works

1. A model with open weights is chosen. The one that answers through a vendor's website cannot be installed in-house: its internals are closed. Open models are used instead — several major global developers release them, and Russian vendors will install their own models inside a customer's perimeter under a separate contract.
2. Hardware is sized to the model. Size is measured in billions of parameters. A 7–8 billion model runs on a single professional graphics card. A 70 billion model needs two to four cards — or it can be compressed: faster and cheaper on hardware, but some accuracy is lost. The choice of model sets the budget, which is why it comes first.
3. The surrounding machinery is assembled. The model itself is only the core. A request queue is needed, a log of queries, access rules by department, and a spare card in case one fails.
4. Company systems are connected — email, the document store, the accounting system.
5. An owner is appointed. Updates, load monitoring, restarts after failures, quality checks after a model version changes. One person alone is a risk: if that person is on holiday or resigns, someone already familiar with the setup has to step in.

Power and cooling form a line item of their own. A server with several graphics cards draws a few kilowatts under load and heats up accordingly; an office server room is rarely built for that, so rack space in a data centre is the usual answer.

One more timeline tends to be discovered late — delivery. Professional graphics cards are seldom in stock: several months typically pass between order and a working server — procurement, import, installation, configuration. In that time a cloud pilot can show whether the project is needed at all and at what volume it pays off.

A case from practice

An engineering firm planned to launch an assistant for internal regulations, and the security department forbade any documents from leaving the perimeter — internal policy said so. The initial budget: a server with four graphics cards, several million rubles to buy, four months to deliver and configure.

Before the purchase, the flow itself was examined: 4,500 documents per month. Restricted-access rules covered 380 of them — eight percent. The rest were instructions, regulations, and standard letters containing no personal data and nothing restricted.

The flow was split. Eight percent stayed on an in-house server, which needed only one graphics card. Everything else went to a Russian cloud billed per query. Purchase costs fell by roughly a factor of four, and the launch happened in three weeks instead of four months.

What the business gains

• The project becomes possible where sending data out is forbidden. Without an in-house perimeter, sign-off from the security department stops the work at the idea stage.
• Cost stops depending on volume. With large, steady flows, fixed hardware costs beat per-query billing. The break-even point fits on one page: multiply the cloud price per query by the monthly volume and compare it with the three-year cost of owning the server, including a share of the engineer's salary. The AI budget calculator shows the difference between cloud and on-premise — it has a dedicated toggle for this.
• The terms cannot be changed from outside. Vendor pricing, service availability from Russia, usage rules — all of it drops off the risk list. Hardware is bought in advance, so volume and value are tested with a cloud pilot before the purchase — and exactly the right configuration is ordered.

When it can be skipped

The obligation to keep data on company-owned hardware is written into law for a narrow set of cases: state secrets, significant critical information infrastructure facilities, and certain state systems. Other prohibitions more often grow out of internal policy.

Federal Law 152-FZ requires something different: the database holding personal data of Russian citizens must be located on Russian territory. A Russian cloud covers that condition.

Anonymisation works too. Strip names, contract numbers, and contact details from the text, and what leaves the perimeter is a document with no personal data in it.

Finally, volume. At a few hundred queries a day the graphics cards sit idle most of the time — and are paid for in full regardless.

Most often the argument is settled by splitting the flow: sensitive documents are handled by an in-house server, everything else goes to a cloud billed per query. That division satisfies the security department and keeps the budget manageable. The place to start is a count: what share of documents genuinely falls under the restriction.

What to check

Ask for the basis of the prohibition in writing: the specific clause of a specific document. Some restrictions were written before Russian AI services existed and have not been revisited since.

Next, the full three-year budget: hardware, electricity, rack space, a share of the engineer's salary, replacement of a failed card. Clarify separately who maintains the system after handover. Models are updated every few months, and a server left untended falls noticeably behind subscription models in capability within a year.

And one last thing: how will anyone know the system has started answering worse? Quality drifts imperceptibly, especially after a model version changes. Set aside a set of fifty real queries with the answers considered correct, and run it after every update. Without that check, the news of declining quality comes from employees — and usually too late.