Hybrid search
Combining lexical BM25 and dense vector retrieval and fusing the two rankings, usually with reciprocal rank fusion (RRF) or a weighted score blend.
Contents
Hybrid search runs a lexical (keyword, usually BM25) search and a dense (embedding) search over the same corpus and merges the two result lists into one ranking. Lexical search matches exact terms: product codes, error strings, names, acronyms. Dense search matches meaning: paraphrases and synonyms. Each catches what the other misses, which is why the owner’s guide recommends hybrid search wherever “exact IDs/keywords matter alongside meaning” (Notebook). In Anthropic’s tests, adding BM25 to contextual embeddings cut retrieval failures from 3.7% to 2.9% (Anthropic).
The two retrievers
BM25 (sparse / lexical). A bag-of-words score summed over query terms (Elastic):
- IDF: rare terms count more than common ones.
- Term-frequency saturation,
k1(default 1.2): repeated occurrences add less and less; lower k1 saturates faster. - Length normalization,
b(default 0.75): long documents are penalized;b = 0turns this off.
It needs no model, is cheap, explainable, and handles out-of-vocabulary strings an embedder never saw. It fails on synonyms and paraphrase.
Dense retrieval. An embedding model maps query and chunk into vectors; an ANN index in a vector database finds nearest neighbours. It captures meaning but can blur exact identifiers. There is also a theoretical ceiling: for a given embedding dimension, only a bounded number of distinct top-k result sets is reachable by any query, and state-of-the-art embedders fail a deliberately simple benchmark built to need many combinations (Weller et al.). Synthesis: that limit is one argument for keeping a non-embedding retriever in the loop.
Fusion
The two retrievers produce scores on incomparable scales (BM25 is unbounded; cosine is in [-1, 1]), so you either fuse by rank or normalize the scores.
flowchart LR
Q["Query"] --> B["BM25 top N"]
Q --> V["Vector kNN top N"]
B --> F{"Fusion"}
V --> F
F -->|"rank-based"| R["RRF: sum 1/(k + rank)"]
F -->|"score-based"| C["Normalize, then alpha*dense + (1-alpha)*lexical"]
R --> O["Fused candidates -> reranker"]
C --> OReciprocal rank fusion (RRF)
For each document, sum 1 / (k + rank) over every list it appears in; k = 60 was fixed in the original paper’s pilot study and never tuned (Cormack et al.). Properties:
- Uses only ranks, so no score normalization and no training data.
- Top ranks dominate, but lower ranks still count;
kdamps a single system that puts an outlier first. - A document found by both retrievers gets two contributions, so agreement is rewarded.
- In the 2009 study it beat every individual system and the Condorcet and CombMNZ fusion methods (Cormack et al.).
Worked example (Synthesis): a doc ranked 1st by BM25 and 3rd by vectors scores 1/61 + 1/63 = 0.0323; a doc ranked 1st by vectors only scores 1/61 = 0.0164. Agreement wins.
Convex combination (weighted scores)
Normalize each score list (e.g. min-max), then alpha * dense + (1 - alpha) * lexical. Bruch et al. found a tuned convex combination beats RRF in and out of domain, that RRF is sensitive to its parameters, and that tuning alpha needs only a small labelled set (Bruch et al.).
Contradiction: Cormack et al. present RRF with a fixed k = 60 as robust and consistently better than alternatives (source); Bruch et al. report RRF is parameter-sensitive and beaten by a tuned convex combination (source). Synthesis: the reconciliation is practical. RRF is the safe zero-training default; a weighted blend wins if you have labelled queries to tune alpha.
In real engines
| Engine | API | Key knobs |
|---|---|---|
| Elasticsearch / OpenSearch-style | rrf retriever wrapping a standard (BM25) and a knn retriever | rank_constant (default 60, higher = flatter), rank_window_size (results per child before fusion; larger is better and slower). No scroll, sort or rescore; paging past the window returns nothing (Elastic docs) |
| Weaviate | hybrid query | alpha (1 = pure vector, 0 = pure keyword), fusionType Ranked vs Relative Score (Relative Score default since v1.24), maxVectorDistance (vector side only), queryProperties with boosts like title^2 (Weaviate docs) |
Synthesis: Postgres can do the same by combining full-text search and pgvector results in SQL and computing RRF yourself; many vector databases now ship built-in sparse + dense fusion.
Learned sparse and multi-vector alternatives
Synthesis: learned sparse models (SPLADE-style term expansion) and multi-vector late interaction (see ColBERT under reranking) sit between BM25 and single-vector dense retrieval. They can replace one side of a hybrid, at the cost of larger indexes.
Failure modes
- Unbalanced candidate pools: fusing top 10 from each side when the right doc is 40th in both. Retrieve wide (Anthropic retrieved 150 before reranking) (Anthropic).
- Score fusion without normalization: BM25’s unbounded scores swamp cosine (Synthesis).
- BM25 on poor tokenization: language analyzers, stemming and stop words decide what “exact match” means; code and IDs may need a custom analyzer (Synthesis).
- Chunks without context hurt both sides; contextual BM25 (indexing the LLM-written context with the chunk) helped in Anthropic’s tests (Anthropic).
When to use / when not
- Use for enterprise and technical corpora full of identifiers, jargon and names; it is a cheap, reliable gain and pairs naturally with a reranker (Notebook).
- Dense only may suffice for conversational, paraphrase-heavy content with few exact identifiers (Synthesis).
- BM25 only is a strong baseline when you can’t host embeddings; measure before assuming dense wins (Synthesis).
Sources
- Elastic: Practical BM25, Part 2
- Cormack, Clarke & Büttcher 2009: Reciprocal Rank Fusion
- Bruch, Gai & Ingber 2022: Fusion Functions for Hybrid Retrieval
- Elasticsearch docs: RRF retriever
- Weaviate docs: Hybrid search
- Anthropic: Introducing Contextual Retrieval
- Weller et al.: Theoretical Limitations of Embedding-Based Retrieval
- Notebook: RAG vs CAG vs GraphRAG