Source: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

Wei et al. (2022): few-shot exemplars with intermediate reasoning steps sharply improve large models on arithmetic, commonsense and symbolic reasoning.

Contents
  • What: research paper (NeurIPS 2022), arXiv:2201.11903
  • Authors: Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou (Google)
  • Date: v1 2022-01-28, v6 2023-01-10
  • Original: arXiv:2201.11903

Summary

Introduces chain-of-thought (CoT) prompting: instead of few-shot exemplars of question → answer, give exemplars of question → reasoning steps → answer. The model then writes its own intermediate steps before answering, which substantially improves accuracy on multi-step problems for sufficiently large models.

Key claims

  • Generating a series of intermediate reasoning steps improves complex reasoning in LLMs.
  • The ability emerges with scale: it shows up in sufficiently large models given a few CoT exemplars.
  • Gains across arithmetic, commonsense and symbolic reasoning on three model families.
  • PaLM 540B with eight CoT exemplars reached state of the art on GSM8K at the time, beating fine-tuned GPT-3 with a verifier.

Feeds

Chain-of-thought · Few-shot prompting

Suggest changes →