Source: MTEB: Massive Text Embedding Benchmark (Muennighoff et al.)

The paper that defined the standard benchmark for text embedding models and showed no single model wins every task.

Contents

Summary

MTEB gathered 58 datasets spanning 8 embedding task types and 112 languages, and benchmarked 33 models on them. Its central finding is that embedding quality is task-dependent: a model that ranks first on one task type need not do so on others. It became the de facto leaderboard for choosing embedding models.

Key claims

  • 8 task types (including clustering, reranking and retrieval), 58 datasets, 112 languages.
  • 33 models evaluated; no method dominates across all tasks.
  • The field had not converged on a universal text embedding method.

Feeds

Embeddings

Suggest changes →