Source: Language Agent Tree Search (LATS, Zhou et al., 2023)
ICML 2024: Monte Carlo Tree Search over ReAct-style trajectories, with LM value functions, self-reflection and environment feedback.
Contents
- What: research paper (ICML 2024, PMLR vol. 235), arXiv:2310.04406
- Authors: Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, Yu-Xiong Wang
- Date: v1 2023-10-06, v3 2024-06-06
- Original: arXiv:2310.04406 · PMLR · code
Summary
A single ReAct trajectory commits to each step greedily and cannot back up. LATS treats each partial reason-act trajectory as a node in a search tree and runs Monte Carlo Tree Search: the LM proposes candidate next steps, an LM-powered value function scores states, environment feedback grounds those scores, and self-reflections on failed branches are fed into later attempts. It unifies reasoning, acting and planning in one framework.
Key claims
- Evaluated on programming, interactive QA, web navigation and math.
- HumanEval pass@1 with GPT-4: 92.7% per the arXiv v3 abstract (the ICML poster page cites a different figure, 94.4%).
- WebShop with GPT-3.5: average score 75.9 without gradient updates, which the authors call comparable to gradient-based fine-tuning.
- Synthesis (not checked against the paper body): tree search needs an environment that can be reset or reverted to explore sibling branches, and costs many more model calls than a single trajectory.