Skip to content
AI News HubLIVE
More
Summary only

On-Policy Distillation: Efficiently Training Specialized Small Models (2025)

Thinking Machines Lab proposes On-Policy Distillation, combining the relevance of RL with the dense reward of distillation to train small models at a fraction of the cost. Experiments show 50-100x compute savings over RL on math reasoning, and effective recovery of instruction-following after domain fine-tuning.

SourceHacker News AIAuthor: nateb2022
On-Policy Distillation: Efficiently Training Specialized Small Models (2025)
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions

Summary only

Under the source content permission, this page provides an editorial summary, key points, and a link to the original article.

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • On-policy distillation samples rollouts from the student and grades each token with a teacher, providing dense feedback.
  • It achieves 50-100x compute efficiency gain over RL on math reasoning tasks.
  • In personalization, it restores instruction-following degraded by domain fine-tuning.
  • Supports continual learning by avoiding catastrophic forgetting seen in off-policy methods.

Highlights and analysis are generated automatically and may contain errors. Check the original source.

More recent stories on this topic · Models