Source: Turning Up the Heat: Min-p Sampling (Nguyen et al., 2025)
Proposes min-p, a truncation sampler whose cut-off scales with the top token's probability, improving quality at high temperature.
Contents
- What: research paper (arXiv 2407.01082, ICLR 2025 oral)
- Author: Minh Nhat Nguyen, Andrew Baker, Clement Neo, Allen Roush, Andreas Kirsch, Ravid Shwartz-Ziv
- Date: 2024-07-01 (v1); revised 2025-11-20 (v8)
- Original: arXiv:2407.01082
Summary
Min-p keeps tokens whose probability is at least a fraction (min_p) of the most likely token’s probability. When the model is confident the pool is tiny; when it is uncertain the pool widens.
Key claims
- Top-p struggles to balance quality and diversity, especially at higher temperatures.
- Min-p sets the threshold relative to the top token’s probability, adapting to model confidence.
- It improved quality and diversity on GPQA, GSM8K and AlpacaEval creative writing for Mistral and Llama 3 models (1B-123B), most at high temperatures.
- Hugging Face Transformers and vLLM have adopted min-p.