Source: On-Policy Distillation of Language Models (GKD)
The 2023 Google DeepMind paper introducing Generalized Knowledge Distillation for autoregressive models.
Contents
- What: research paper (arXiv:2306.13649, ICLR 2024)
- Author: Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, Olivier Bachem
- Date: 2023-06-23 (v3 2024-01-17)
- Original: arXiv:2306.13649
Summary
Identifies the train–inference mismatch in standard sequence distillation and fixes it by training the student on its own generated sequences with teacher feedback, using a flexible choice of divergence.
Key claims
- Standard KD for autoregressive models trains on sequences the student will not itself produce at inference.
- GKD trains on student-generated sequences scored by the teacher.
- Alternative divergences help when the student cannot represent the teacher’s full distribution.
- Can be combined with RL fine-tuning; evaluated on summarisation, translation, arithmetic reasoning and instruction tuning.