What Semantic Search Is and What It Does for a Business

Why this matters

• Understand why a knowledge base fails to answer questions whose answers are sitting inside it
• Estimate what share of support requests can be closed by search without a human
• Ask a vendor how search quality is measured on the company's own documents

What semantic search is, in plain language

At an auto parts shop a customer asks for "windshield blades." In the catalogue the item is called "frameless wiper, 600 mm." Zero words in common. The salesperson finds the right box in a second, because they understand what was meant.

Keyword search cannot do that. It matches letters and word forms, so it finds a document only when the person asking has already guessed the wording used in the text.

Semantic search hands the machine exactly the job the experienced salesperson does. Every chunk of text is converted in advance into a set of numbers — an embedding. Texts close in meaning get numbers close to each other. From there the numbers are what gets compared, and matching words stop being a requirement.

How it works

The first three steps happen once and are repeated whenever documents are updated. The last two take a fraction of a second per query.

Then the path forks. The retrieved chunks are shown to a person as a list — that gives smart search across the knowledge base. Or they are passed to a language model so it can assemble a coherent answer with links to the source: that scenario is called RAG.

The storage itself is a vector database. Up to a few hundred thousand chunks, an extension to the ordinary database the company already runs will do the job.

An example from practice

A service company keeps equipment manuals in a shared folder: roughly a thousand files accumulated over ten years. An engineer on site calls the office, because finding the right paragraph by filename search is slower than picking up the phone. Every such call ties up two people.

The documents are split into chunks, vectors are computed, and search by meaning goes live. The query "won't start after sitting idle in winter" finds the section on warming up the hydraulics — the word "winter" appears nowhere in the manual.

Quality is measured before launch and after. A hundred requests are pulled from last quarter's call log, the correct document is marked for each, and the share where it landed in the top five results is calculated. The same list is run again after every adjustment to chunking. If the share rises, the adjustment was worth making.

And here is the honest limitation: search by work order number and by serial number stays as it was, on exact matching. A number carries no meaning, so there is nothing to compare.

What it does for a business

• Answers arrive without a middleman. Questions that previously sent an employee to support or to a colleague get resolved on their own. The freed-up hours should be assigned in advance — otherwise the savings stay on paper.
• Knowledge stops living inside people. Policies, correspondence and accumulated solutions become available to a newcomer from day one. Time to productivity shortens, and the request log makes that measurable.
• The knowledge base comes alive without a rewrite. Documents stay where they are, in the format they are in. The effort goes into chunking and verification, not into tagging every file by hand.
• It opens the road to model-generated answers. Search by meaning is a mandatory part of RAG: the model answers from company documents and cites the source. Without search, the model answers from memory, and memory is not always accurate.

When it can be skipped

A small base is served well enough by ordinary search. Fifty documents an employee knows by heart, and preparing vectors has nothing to pay for itself with.

Search by strict identifiers is also best left alone. Contract numbers, part numbers, tax IDs, dates — these call for an exact match, and computing closeness only gets in the way. The working solution is usually hybrid: exact search for numbers, semantic search for phrasing.

A separate case is documents that exist only on paper or that have gone out of date. Search by meaning will quickly surface an obsolete policy and present it as current. Order is restored in the sources first, and search is connected after.

What to check

First: how the documents are chunked. Chunks that are too small lose context; chunks that are too large blur the meaning and drag irrelevant material into the answer. Chunk boundaries are better drawn along the document's own sections than by character count. This can be checked by eye — read twenty random chunks and see whether each one stands on its own.

Second: what computes the vectors for the language in use. Models differ, and the difference shows up on one and the same set of queries. The comparison is only meaningful on the company's own documents, not on the description in a model card. It is worth reviewing the weak spots of embeddings up front: negation, numbers and very long fragments come through worse than anything else.

Third: what happens when documents are updated. A policy changes — the chunks are recomputed, or search will confidently keep serving the old edition. The update procedure is written down before launch, along with the person responsible for it.