The World Inside Neural Networks

This article explores the geometric structure of internal representations in neural networks, termed 'neural geometry.' The authors argue that concepts are encoded as curved manifolds rather than linear directions, and understanding this geometry is crucial for precise interpretability and control. Examples from mountain car simulation, month cycles, and slant rhymes illustrate the concept, while limitations of sparse autoencoders are discussed. The article concludes that neural geometry is a key frontier for safer and better AI.