Source: Tantivy bm25.rs (source code)
Tantivy's BM25 scorer: hard-coded k1 = 1.2 and b = 0.75, Lucene-style IDF, (k1+1) factor, and a 256-entry fieldnorm cache.
Contents
- What: open-source Rust library source code and API docs (tantivy 0.27.0 on docs.rs)
- Author: Quickwit / tantivy contributors
- Date: read 2026-11-14
- Original: src/query/bm25.rs · Bm25Weight on docs.rs
Summary
Tantivy is a Lucene-inspired full-text search library in Rust. Its BM25 weight is built from collection statistics supplied by a Bm25StatisticsProvider (by default the Searcher, summing over segments).
Key claims
- k1 = 1.2 and b = 0.75 are compile-time constants; the public constructors take no k1 or b, only term statistics and an optional multiplicative boost.
- IDF = ln(1 + (N - n + 0.5)/(n + 0.5)), the same form as Lucene.
- The explanation output includes a (K1 + 1) factor in the tf component, unlike current Lucene; rankings are unaffected.
- Document length is a one-byte fieldnorm id; the length-normalization term is cached for all 256 ids.
- Statistics (total tokens, document count, document frequency) can be overridden by implementing the provider trait, e.g. to share statistics across indexes.