Source: Günther et al. 2024: Late Chunking

Jina AI paper that embeds the whole document first and pools token vectors per chunk afterwards, so chunks keep document context.

Contents

Summary

Proposes reversing the usual order of chunk-then-embed: run a long-context embedding model over the full text to get token embeddings, then mean-pool them within each chunk’s span.

Key claims

  • Each chunk embedding then reflects surrounding document context (e.g. what a pronoun refers to).
  • Needs no extra training; a fine-tuning recipe further improves it.
  • Reported better retrieval than naive chunking across several tasks.
  • Requires a long-context embedding model with access to token-level outputs (not just a pooled API vector).

Feeds

Chunking

Suggest changes →