Vector Databases

Systems that store embeddings with metadata and answer filtered nearest-neighbour queries at scale; how pgvector, Pinecone, Weaviate, Qdrant, Milvus and Chroma differ.

Contents

A vector database stores embeddings alongside their source data and metadata, and answers “which stored vectors are nearest to this query vector, subject to these filters?” in milliseconds over millions to billions of items. It is the storage and retrieval layer of RAG. The hard parts are not the similarity math but everything around it: approximate indexes, filtering, updates, tenancy, memory cost and operations (notebook).

Data model and request path

Every product shares the same vocabulary: a collection (index, namespace, table) of fixed-dimension vectors; a distance metric fixed at creation; a payload of metadata per vector; upsert by ID as the main write; top-k queries with optional metadata filters (notebook).

Synthesis: the query path below is common to all of them; products differ in where filtering happens and where data lives (RAM, SSD, object storage).

flowchart LR
  Q["Query text"] --> E["Embed<br/>(same model as corpus)"]
  E --> P["Planner: estimate<br/>filter selectivity"]
  P -->|"broad filter"| A["ANN index search<br/>(HNSW / IVF / DiskANN)<br/>filter during traversal"]
  P -->|"very selective filter"| F["Exact scan over<br/>matching IDs"]
  A --> R["Optional rescoring<br/>with full-precision vectors"]
  F --> R
  R --> K["Top-k IDs + payload"]
  K --> H["Optional: fuse with BM25,<br/>rerank, then LLM"]

Filtering is the real differentiator

Approximate indexes only find the near neighbours they visit. Filtering after the index search starves results: in pgvector, an HNSW scan with ef_search 40 and a filter matching 10% of rows returns about 4 rows on average (pgvector). Systems handle this differently:

  • pgvector: post-filters by default. Since 0.8.0, iterative index scans (hnsw.iterative_scan = strict_order | relaxed_order, capped by hnsw.max_scan_tuples, default 20,000) keep scanning until enough rows pass. For selective filters, use a B-tree on the filter column, partial indexes, or partitioning (pgvector).
  • Qdrant: filterable HNSW adds extra graph edges per indexed payload value so filtered traversal stays connected. A query planner switches to full scan when the estimated match set is below full_scan_threshold (10,000 KB of vectors by default), and ACORN (v1.16, opt-in per query) explores neighbours of neighbours for strict filters, trading speed for accuracy. Payload indexes must exist before the HNSW build or the extra edges are missing (Qdrant).
  • Weaviate: builds an allow-list from its inverted index first, then traverses HNSW with the ACORN strategy (default since v1.34). It drops to flat search over matches when the filter is very restrictive, around 15% of the dataset (Weaviate).
  • Milvus: recommends picking the index by filter ratio (share of data filtered out): graph below 85%, IVF between 85% and 95%, brute-force FLAT for very high ratios (the page says over 98% in its prose and over 95% in its decision matrix, and calls these rules not always correct) (Milvus).
  • Faiss (library): ID-selector callbacks, with metadata-first brute force chosen when the subset is tiny (Faiss).

Product trade-offs (as of November 2026)

StoreShapeIndex optionsFilteringScale / ops profileBest fit
pgvectorPostgres extension (0.8.x)HNSW, IVFFlat; vector / halfvec / bit / sparsevecSQL WHERE; post-filter plus iterative scansPostgres operations, joins and transactions; indexable dims max 2,000 (vector) or 4,000 (halfvec)Already on Postgres; vectors next to relational data
PineconeManaged serverlessProprietary geometric partitioning plus freshness layerMetadata filters; namespaces as hard partitionsNo infra; storage on blob, compute caches segments; cold queries take secondsHands-off production, many tenants
WeaviateOSS + cloudHNSW, flat, dynamic, HFresh (disk, preview in v1.36, GA in v1.38); PQ / BQ / SQ / RQAllow-list + ACORNBuilt-in vectorizers, hybrid search, multi-tenancyBatteries-included hybrid search
QdrantOSS + cloud (Rust)HNSW, sparse inverted index; quantizationFilterable HNSW + planner + ACORNTenant-aware payload indexes (is_tenant)Heavy metadata filtering, self-hosting
Milvus / ZillizOSS distributed + cloudAlmost every family: FLAT, IVF*, HNSW*, DiskANN, ScaNN, RaBitQ, GPU CAGRAScalar filtering; index choice by filter ratioBillion-scale, more components to runVery large corpora, GPU, dedicated team
ChromaOSS (Apache 2.0) embedded/server + Chroma CloudDense, sparse and hybrid searchMetadata, keyword and regexSimplest local startPrototypes, notebooks, small/medium apps

Sources: pgvector, Pinecone, Weaviate, Qdrant, Milvus, Chroma, notebook.

Architecture notes worth knowing

  • Pinecone serverless writes to a log and blob storage (the source of truth). An index builder partitions vectors around centroids into immutable segments, and a small freshness index makes new writes searchable within seconds. Executors cache segments, so infrequently queried namespaces are cheap but their first query can take about a couple of seconds, up to about 20 s at billion scale (Pinecone).
  • Weaviate’s dynamic index starts flat and switches one way to HNSW at 10,000 objects, so small tenants never pay for a graph (Weaviate).
  • pgvector HNSW builds are far faster when the graph fits in maintenance_work_mem, and can be parallelized. IVFFlat must be built after loading data because its clusters are trained on it (pgvector).
  • FAISS is a library, not a database: you own persistence, metadata and updates (notebook).

Multitenancy

Options run from hard to soft isolation: separate collections or tables; Pinecone namespaces (hard partitions, hundreds of thousands per index) (Pinecone); Postgres list partitioning (pgvector); Qdrant is_tenant payload indexes that co-locate a tenant’s data (Qdrant); Weaviate per-tenant shards with flat-to-HNSW dynamic indexes (Weaviate). Synthesis: a shared index with a tenant_id filter is the cheapest option but the leakiest. One forgotten filter exposes another customer’s data, so enforce it server-side.

Cost and memory

Raw float32 vectors cost 4 × dims bytes, and a graph index adds link storage on top. Milvus estimates HNSW at 640 MB for 1M 128-dim vectors, versus 136 MB with PQ (Milvus). Storing embeddings as int8 or binary cuts storage 4x or 32x; 250M 1024-dim vectors drop from about 954 GB to about 30 GB as binary (HF). pgvector supports halfvec and binary-quantized expression indexes with re-ranking (pgvector). See ANN indexes for the compression options.

Failure modes

  • Filtered queries returning too few results, from post-filtering or payload indexes created after the graph (pgvector, Qdrant).
  • Metric or dimension mismatch with the embedding model (notebook).
  • Model upgrade means full re-embed and re-index. Synthesis: plan a dual-write or blue/green collection swap.
  • Deletes and updates degrade graphs. HNSW in Faiss does not support removal (Faiss), and Qdrant suggests its ACORN mode when many points are deleted (Qdrant).
  • Cold-start latency on serverless tiers (Pinecone).
  • No keyword recall unless hybrid search is enabled (notebook). See hybrid search.

When to use which

Synthesis, built on the notebook’s rules of thumb (notebook):

  • Prototype: Chroma or in-process FAISS; the chunk, embed, upsert and query flow does not change when you swap later.
  • Already run Postgres, under roughly 10M vectors, need joins and transactions: pgvector. Turn on iterative scans for filtered queries.
  • Managed, spiky or multi-tenant workloads, no ops team: Pinecone, or Qdrant, Weaviate or Zilliz cloud.
  • Heavy metadata filtering, self-hosted: Qdrant or Weaviate.
  • Billions of vectors, SSD or GPU indexes: Milvus, or anything offering DiskANN.
  • Do you need one at all? Under about 10k vectors, brute force is competitive (Faiss), and a small corpus may fit in the context window with prompt caching.

Sources

Suggest changes →