If you are building AI search or retrieval, the assumed next step is to add a vector database. Sometimes that is right. Often it is infrastructure a project takes on before it has the problem the infrastructure solves. It is worth asking whether you need one yet.
What a vector database is for
AI features find relevant text by comparing numerical representations of meaning — embeddings — and returning the closest matches. A vector database is built to do that comparison quickly across a very large number of items. That is a real capability, and at large scale it is genuinely useful.
Your existing database may already do it
Here is the part that gets skipped: many relational databases can now store and search embeddings directly. For a corpus of thousands or even modest millions of items, that is frequently fast enough — and it keeps everything in one system you already run, back up, and understand. No new service, no extra thing to operate.
- Thousands to low millions of items — your existing database with vector support is often enough.
- Tens of millions and up, with tight latency — a dedicated vector database starts to earn its keep.
- Unsure — start with what you have and measure; adding specialised infrastructure later is easier than removing it.
Every new datastore has a running cost
A separate vector database is another service to deploy, secure, back up, monitor, and keep in sync with your source of truth. That last part is the quiet one: now the same data lives in two places and can drift. None of it is unmanageable, but it is real work, and it should be justified by a need you actually have.
Reach for a vector database when your scale demands it — not because it is the expected box on the diagram.
Retrieval quality beats the datastore choice
Whichever you use, the thing that decides whether AI search is good is the quality of retrieval — how you chunk, what you embed, and how you rank results. A dedicated vector database makes search faster at scale; it does not make bad retrieval good. Spend your effort there first, and let the scale you actually reach decide when the infrastructure needs to grow.