Source: Asai et al. 2023: Self-RAG

Paper training an LLM to emit reflection tokens that decide when to retrieve and critique relevance and support.

Contents

Summary

Trains a single generator to output special reflection tokens alongside text, making retrieval on-demand and letting decoding favour well-supported segments.

Key claims

  • Reflection tokens: Retrieve (yes / no / continue), IsRel (relevant / irrelevant), IsSup (fully / partially / no support), IsUse (1-5).
  • A Llama 2 7B critic, distilled from GPT-4 labels (4k-20k examples per token type, over 90% agreement), inserts reflection tokens into 150k training examples offline; the generator learns them with ordinary next-token loss.
  • Inference: retrieve when P(Retrieve=yes) exceeds a threshold (default 0.2); segment-level beam search (width 2) scores segments by likelihood plus weighted IsRel, IsSup, IsUse (1.0, 1.0, 0.5).
  • 7B and 13B Self-RAG models beat ChatGPT and retrieval-augmented Llama2-chat on open-domain QA, reasoning and fact verification, with better citation accuracy.

Feeds

Agentic RAG

Suggest changes →