What Is a Vector Database and Why Business Needs It

Why this matters

• Understand how an AI system finds the right document among thousands without going through them all
• Judge whether separate infrastructure is needed or an extension to the existing database will do
• Ask a vendor the right questions: how many documents, which embedding model, how the database gets updated

What is a vector database in plain terms

Picture a library where books are shelved not alphabetically or by genre, but by what is inside them. Agatha Christie's detective novels sit next to Conan Doyle's. Gardening books sit next to landscape design. To find something like The Hound of the Baskervilles, there is no need to know the author or the exact title — walking up to the English detective shelf is enough.

A vector database is built on the same idea. Every document gets coordinates in a space of meaning — hundreds or thousands of numbers describing what the text is about. Documents with similar content end up near each other. Search becomes a question of what lies closest to a given point.

Hence the point that matters for business. Ordinary keyword search will not find a "lease contract" if the document says "rental agreement". Vector search will: both texts land next to each other in the space of meaning. That is what makes it possible to build systems that understand questions asked in plain language.

How it works

1. Documents are cut into pieces. A forty-page contract is split into fragments of a paragraph or a few hundred words. Chunk size is tuned to the task: too small and context is lost, too large and search precision suffers.

2. Each piece is turned into a vector. An embedding model reads the text and produces a list of numbers — typically 384 to 1,536 coordinates. Those numbers encode meaning: texts about the same thing get similar sets of numbers.

3. The vectors go into the database. The original text and its metadata are stored alongside the vector: document title, date, department. Metadata can then be used as a filter — searching only contracts, or only last year's records.

4. At query time the question is turned into a vector too. The same embedding model processes the user's question and produces its coordinates in the same space.

5. The database returns the nearest neighbours. The algorithm finds the fragments whose coordinates sit closest to those of the question. Usually the three to ten closest are taken.

6. The retrieved fragments go to the model. A language model receives the user's question along with the relevant document chunks and builds an answer from them. This pairing is known as RAG — retrieval-augmented generation.

A case from practice

A logistics company kept its procedures, instructions and frequently asked questions in a corporate wiki — some 8,000 pages. Operators spent 4–7 minutes hunting for the right passage: it meant guessing the exact words the author of the document had used.

The documents were loaded into a vector database. Each page was split into chunks of 400–600 words, producing roughly 35,000 fragments. The embedding model processed them in a few hours, and the one-off cost of creating the vectors came to about RUB 4,000.

An operator now types a question in ordinary language: "what to do if a shipment is held at customs". The database returns three to five relevant fragments in a fraction of a second. Time spent looking for information fell to 40–90 seconds. Measurements were taken on a sample of 200 enquiries in the month before launch and the month after: average time to first reply to a customer dropped from 11 minutes to 6.

What business gets out of it

• The model answers from company documents rather than general knowledge. Without a vector database, a language model knows only what it was trained on. With one, it receives current internal documents and can cite specific clauses of a procedure.

• The risk of hallucination drops. The model sees the source and builds its answer on it instead of inventing. In systems with vector search, answers can be checked: the fragments they were assembled from are displayed alongside.

• The token budget shrinks. Rather than sending the model every document, only the relevant ones are sent. That cuts input volume several times over, and the supplier's invoice with it.

• Search works in natural language. The user does not need to know the exact terms used in the document. The question "how do I process a return" will find an instruction that says "reverse goods acceptance procedure".

When it can be skipped

With a small volume of documents, a vector database is overkill. If the corporate knowledge base runs to 50–100 pages, all of it fits inside the context window of a modern model. Search by meaning is then handled by the model itself, and no separate infrastructure is needed.

The same goes for tasks where exact matching matters more than semantic closeness. Searching by contract number, product code or tax ID is more reliable done the classic way — on an index in a conventional database.

A vector database will not help either where documents have not been digitised or are stored as scans with no text recognition. An embedding model works with text, not pictures. Recognition has to be solved first, vector search second.

What to check

First: search quality depends on the embedding model. The same database gives different accuracy with different models. Before launch it is tested on real questions: 30–50 typical queries are put to it and the top three results are checked for relevance. If they miss, the model or the chunking parameters get changed.

Second: the database needs updating. New documents have to be turned into vectors and added, outdated ones removed. Without that, within six months the system starts answering from obsolete procedures. The update process belongs in the design stage.

Third: a vendor is worth asking how many documents are supported, what the initial load and the updates cost, and where the data physically sits. For sensitive documents, it matters that the vector database runs on the company's own servers rather than in a foreign cloud.