Source: Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs (MIPRO, 2024)
The MIPRO paper: jointly optimizing per-module instructions and few-shot demos in LM programs without module-level labels, using grounded proposals and a surrogate model.
Contents
- What: research paper (arXiv 2406.11695; EMNLP 2024)
- Authors: Krista Opsahl-Ong, Michael J. Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, Omar Khattab
- Date: submitted 2024-06-17
- Original: arxiv.org/abs/2406.11695
Summary
Prompt optimization for multi-stage LM programs must work with only a downstream metric: no per-module labels and no gradients. MIPRO factorizes the problem into each module’s free-form instruction and few-shot demonstrations, then searches over combinations.
Key claims
- Strategies: program- and data-aware instruction proposals; a stochastic mini-batch evaluation used to fit a surrogate model of the objective; and meta-optimization of how proposals are generated.
- MIPRO beat baseline optimizers on five of seven multi-stage programs using Llama-3-8B, by up to 13% accuracy.
- Released in DSPy (as
MIPROv2).