Source: Zheng et al. 2023: Step-Back Prompting

ICLR 2024 paper (Google DeepMind) that has the model first ask a more abstract 'step-back' question, then reason or retrieve from the principles it surfaces.

Contents

Summary

Proposes a two-step prompt: first ask the model to abstract the specific question into a higher-level concept or principle (the “step-back question”), then answer the original question grounded in that abstraction. In the knowledge-QA setting, the step-back question is also used for retrieval, so the retriever fetches background facts that the literal question would not match.

Key claims

  • Two stages: abstraction (derive high-level concepts or first principles) and reasoning (solve the original question guided by them).
  • Evaluated on PaLM-2L, GPT-4 and Llama2-70B across STEM, knowledge QA and multi-hop reasoning.
  • Reported PaLM-2L gains: MMLU Physics +7%, MMLU Chemistry +11%, TimeQA +27%, MuSiQue +7%.

Feeds

Query transformation

Suggest changes →