What is deep learning and where it pays off

Why this matters

• Knowing exactly what the invoice covers: data labelling, training hours, or every single run of the model
• Telling apart a task that hand-written rules can handle from one that needs examples
• Asking the vendor for the quality metric and the sample it was measured on

What is deep learning in plain words

Goods receiving at a warehouse is laid out as a series of desks. At the first, the box is checked as a whole: intact, right size. At the second, the label is read. At the third, it is matched against the delivery note. Each desk receives the conclusion of the one before it.

A neural network with deep learning is built along the same lines. It has dozens of layers, sometimes hundreds. The early ones catch simple things: edges, strokes, common letter combinations. The later ones assemble those into a signature, a stamp, the meaning of a phrase.

There is one difference from the warehouse. Nobody writes the instructions for each desk in words. They are worked out from examples with known answers — thousands of them, and millions when training starts from scratch.

Hence the word "deep". It refers to the number of layers, not to depth of thought.

How it works

1. Examples with answers are collected. A thousand scanned delivery notes, each with the required fields filled in by hand. These are labelled data, and they usually cost more than everything else in the project.
2. An example is passed through the layers. The model produces its own version of the answer. At the start it is random.
3. The error is measured and the coefficients adjusted. The gap from the correct answer is passed back through the layers, and each layer shifts its numbers towards a smaller error. That is what training is.
4. This is repeated many times. Here is where the graphics-card hours and the money to rent them go.
5. Testing happens on a held-out sample. Some examples are hidden from training in advance. Quality is measured only on those: on data it has already seen, the model will look better than it really is.

Training from scratch is rarely necessary today. A ready-made pre-trained model is taken and fine-tuned for the task at hand. The volume drops from millions of examples to hundreds, and the invoice drops with it.

An example from practice

Delivery-note processing at a trading company. Some documents arrive as scans: a stamp sits across the quantity column, the scan is at an angle, some rows are filled in by hand. Rules based on field coordinates break on the very first new supplier form.

The task is handled by a model trained on labelled scans. The labelling is done by hand: documents from the past quarter are taken and the fields filled in. Then the share of documents that pass without operator corrections is counted.

The measurement method is the same for every project of this kind. A sample of several hundred scans is set aside before training and left untouched. Quality on that sample is compared with the same figure for people over the same period. The threshold is set by the business: for instance, the share of documents sent for manual checking must not exceed a stated number. Below the threshold, the system is not switched on for the full flow.

What the business gets

• Tasks with no rules become solvable. Reading a skewed scan, parsing the free text of a request, sorting enquiries by meaning — wherever a feature cannot be put into words, deep learning works where written conditions are powerless.
• Quality grows along with the data. An accumulated archive of documents and correspondence turns into an asset: every new thousand labelled examples lifts accuracy, and that gain is measurable.
• The barrier to entry has fallen. Pre-trained models remove the most expensive part — training from scratch. What remains is fine-tuning on in-house examples and renting compute by the hour rather than by the month.
• Costs become calculable. Labelling is counted in labeller hours, training in graphics-card hours, operation in model runs. All three lines can be estimated before the start.

When it can be skipped

The data is already structured — deep learning is not needed. An export from an accounting system, with fields laid out in columns, is processed by ordinary code faster, more cheaply and with a result anyone can follow.

The rules are few and stable — easier to write them out as conditions. A process like that runs for years without retraining and needs neither labelling nor graphics cards.

Volumes run to dozens of cases a month — the saving will not cover the labelling. Whether a particular process is ready for automation is easy to gauge with a checklist before any conversation with vendors begins.

What is worth checking

First: which sample the quality was measured on. A figure obtained on data the model saw during training is always inflated. Ask for the held-out sample, its size, and the period it was drawn from.

Second: what the labelling costs. It rarely appears in the first proposal, yet in practice it eats a noticeable share of the budget. The sum is simple: number of examples times minutes per document times the labeller's rate.

Third: what happens when a new form or a new type of enquiry appears. A trained model needs a documented fine-tuning procedure; without one, quality will drift within six months and there will be nothing to fix it with.