Vector Databases and Approximate Search
Find the nearest vectors among millions without comparing against all of them, using IVF and HNSW indexes and their speed versus recall trade-off.
The problem
After embedding, every document chunk is a vector, and a question becomes a vector too. Retrieval means: find the k stored vectors nearest to the query vector. This is nearest-neighbour search.
The exact method, brute force (also called a flat index), compares the query with every vector. It is simple and perfectly accurate. Its cost grows linearly with the number of vectors.
Estimate
A collection has 1 million vectors with 768 dimensions each, stored as 32-bit floats (4 bytes). How much memory do the raw vectors take?
GB
Estimate
Brute force must read all 3.1 GB for each query. If memory delivers about 10 GB/s to one core, roughly how long does one query take?
ms
At millions of vectors, 300 ms per query is too slow for a live product. We accept approximate nearest neighbour (ANN) search: it may miss a true neighbour in exchange for looking at far fewer vectors.
1 of 6