Source: Chen et al. 2024: BGE-M3 (M3-Embedding)

BAAI paper on one embedding model that emits dense, sparse and multi-vector (late-interaction) representations, multilingual, up to 8,192 tokens.

Contents

Summary

A single model (released as BGE-M3) that can serve dense retrieval, sparse lexical retrieval and multi-vector retrieval from one forward pass, trained with self-knowledge distillation where the combined scores of the three modes teach each mode.

Key claims

  • Three retrieval functions: dense, multi-vector and sparse.
  • Over 100 working languages; inputs from sentences up to 8,192-token documents.
  • Self-knowledge distillation uses relevance scores from the different modes as a teacher signal.
  • Reported state-of-the-art results on multilingual, cross-lingual and long-document retrieval at release.

Feeds

Late-interaction retrieval

Suggest changes →