A brief history of distillation in AI

Knowledge distillation is a model compression technique where a student model mimics a teacher model's outputs, reducing size while preserving performance. This article traces its evolution from Hinton et al. (2015) to modern applications.