The Sequence Knowledge - Issue #954: Learning RSI: How AI Learns to Design Learning Algorithms

TLDR: Most AI systems get better at tasks without getting any better at learning. DeepMind’s DiscoRL goes after that machinery itself: a neural network represents the update rule, and a population of agents supplies the experience used to discover rules that produce better learners. Its Disco57 rule, discovered on Atari57, trained fresh agents on ProcGen and DMLab-30 without further meta-training, a concrete case of intelligence being applied to the procedure that creates intelligence. We trace the lineage from MAML, learned optimizers, and Learned Policy Gradient, then unpack the two-level credit-assignment problem inside DiscoRL’s loop, what zero-shot transfer of a learning rule really tests, why the economics come down to amortizing discovery cost across many training runs, and why interpretability matters before anyone adopts a learned rule. We close with what a genuinely recursive next step would have to show: discovered improvements that keep producing measurable gains after accounting for compute and evaluation effort. An AI system can become a better chess player without becoming any better at learning chess. Give it more training, and its moves improve. The machinery converting experience into improvement can remain unchanged. Recursive self-improvement asks us to examine that machinery. What happens when the learning algorithm itself becomes something a machine can learn? DeepMind’s DiscoRL provides a concrete experiment. A neural network represents an update rule, and a population of agents supplies experience for discovering rules that produce better learners. The resulting algorithm performs strongly across multiple reinforcement learning benchmarks, including environments outside its discovery set. This is a useful entry point into a deeper idea: intelligence can be applied to the procedure that creates intelligence. The part of learning we usually write ourselves Read more