RAG & retrieval
What is a vector database, and do I need one?
A vector database stores high-dimensional embeddings and searches them by similarity rather than by exact match. It uses approximate nearest-neighbour indexing to return the closest vectors in milliseconds, which is what makes embeddings and RAG practical at scale.
A vector database is a specialised database that stores high-dimensional embeddings and searches them by meaning rather than by exact match.[1] You give it a query vector; it gives you back the vectors that sit closest to it. That single operation, nearest-neighbour search over millions of vectors, is the part a normal database is not built to do well.
It matters because of scale. Comparing a query against every stored vector is exact but slow once you have millions of them. So most vector databases use approximate nearest-neighbour (ANN) algorithms — commonly HNSW, a graph-based index that lets a search hop toward nearby vectors instead of scanning every one — which “trade off a bit of accuracy for a huge gain in speed.”[2] The index narrows the search to a handful of likely candidates instead of checking everything, and the result comes back in milliseconds. Pgvector, Weaviate, and Qdrant all support HNSW indexing out of the box, which is part of why it has become close to a default choice rather than a specialized one. You lose a sliver of precision and almost never notice.
This is the storage layer underneath retrieval-augmented generation. The two phases of RAG are: embed and store your documents once, then at question time embed the query and pull the closest chunks. The vector database is where those chunks live and how the second step actually happens. Without it, “find the passages most relevant to this question” has nowhere to run.
So do you need one? Not always.
A few thousand vectors fit in memory and you can score them directly. The case for a real database shows up when the collection grows, when you want metadata filters alongside similarity, or when you would rather not hand-roll an index. Picking among the options is its own question, covered in picking an embedding model, since the model and the store have to agree on dimensions.
Common vector databases
- pgvector ↗
A Postgres extension that adds vector similarity search to a database you may already run.
- Pinecone ↗
A fully managed, cloud-hosted vector database aimed at large scale and low operational overhead.
- Weaviate ↗
An open-source vector database with built-in modules for hybrid keyword-plus-vector search.
- Qdrant ↗
An open-source vector database written in Rust, with strong metadata filtering.
- Chroma ↗
A lightweight open-source store popular for prototyping and small RAG apps.
- Milvus ↗
An open-source vector database built for very large, distributed workloads.
References
- What is a Vector Database? — Elastic
- Vector Search Explained — Weaviate
- pgvector: open-source vector similarity search for Postgres — pgvector
Common questions
- Vector database vs a regular database — what's the difference?
- A regular database finds rows by exact match or range: this id, that date, prices under £10. A vector database finds rows by closeness in meaning. You hand it a query vector and it returns the nearest vectors, ranked by similarity, even when no field matches exactly. That is a different operation, so it needs a different index.
- Do I need a vector database, or just an embedding model?
- An embedding model turns text into vectors. A vector database stores those vectors and searches them. For a few thousand items you can keep the vectors in memory and compare them directly, no database needed. Once you have hundreds of thousands or millions, and you want fast search with metadata filters, a vector database earns its place.