The Equations of Reasoning – Pathway AI

Today’s AI is an engineering miracle that lacks an underlying science: it works remarkably well, yet we cannot fully explain why. When things go off-script, like with noisy input, fuzzy goals, long chains of logic beyond the conditions covered by benchmarks, we do not know what to expect and lack a direct way of fixing the model. In essence, the AI community built steam engines for thought before developing the thermodynamics of intelligence. This blog argues that the missing piece is an axiomatics of artificial intelligence (AI): a small set of laws, in the spirit of statistical mechanics, that derive how a system reasons from how its parts interact. For AI, the microscopic description consists of the elementary operations performed during inference: how computational units activate, communicate, and modify the system’s state. The macroscopic description consists of memory, abstraction, adaptation, and reasoning over time. Connecting these scales would reveal the conditions under which these capabilities emerge and remain stable as the number of units and duration of inference increase. This would make it possible to study limits and failure regimes without retraining and testing every system at every relevant scale, predicting behavior beyond the benchmarks. To address this gap, we introduce The Equations of Reasoning developed in the BDH paper. They establish a formal micro-foundation for Transformer-like reasoning, establishing a rigorous bridge between fast weights and attention on the one hand, and local synaptic dynamics on the other. This bridge lets us observe reasoning locally, through the neurons that information activates and the synaptic states it updates to encode attention and memory. By expressing reasoning as explicit local dynamics, the Equations of Reasoning enable a more rigorous, principle-based approach to AI development. When reasoning is expressed as explicit local-dynamics, long-horizon behavior, continual adaptation, and model composition are all within the reach of mathematical analysis. The Equations of Reasoning thus open a path toward systems whose capabilities, stability, and failure regimes can be characterized before deployment, making safety a design property to be analyzed and engineered, rather than an outcome assessed only after deployment. 1. The missing science of AI The current state of AI can be compared to heat-engine engineering before thermodynamics. Engineers could build increasingly effective engines and compare their fuel consumption, but lacked universal laws that applied across all designs. Carnot’s ideal cycle (a thought experiment describing a perfectly efficient engine) changed the level of analysis: it established a universal upper bound on the efficiency of any engine operating between two temperatures, including engines that had never been built. A theory of intelligence should serve the same function. It should let us derive the possible, stable, and impossible behaviors of a reasoning system from its main parameters and local computations, rather than discover them one architecture and one benchmark at a time. The empirical paradigm has produced great systems, but it has also left AI development largely phenomenological: we observe what works, measure it on benchmarks, and scale it further, without a general theory that explains why a system behaves as it does, or predicts how it will behave outside familiar conditions. For almost a decade after deep learning’s early breakthroughs, we found the answer for AI improvement in brute-force scaling: more data, bigger models, and more compute generally led to better results. Introduced in 2017, the Transformer was especially well suited to this approach. Unlike recurrent networks, the Transformer relies primarily on large matrix multiplications that can run in parallel on GPUs and other accelerators originally built for graphics and dense numerical computation. That fit gave the Transformer a “hardware lottery” advantage, and steered the field toward a brute-force scaling paradigm. The result is a field governed more by measured regularities than by general laws. We observe benchmark results, scaling curves, and internal behaviors [Anthropic, 2023], then infer empirical regularities from them. The field produces hundreds of ideas demonstrated at small scale [Coconut 2024, Atlas 2025, etc.], yet publicly released leading architectures remain variations on a narrow family of designs [Architecture Gallery], mostly changing attention mechanisms, activation functions, or the proportion of active experts in Mixture-of-Experts architectures. Meanwhile, scaling laws are repeatedly challenged and refined as new regimes are measured. This empirical record is indispensable for iterative improvement, but it is far from providing a mechanics of intelligence. A loss function tells us what training rewarded. Gradient flow describes how parameters changed during training. A scaling law summarizes how aggregate performance varies with data, compute, and model size. An interpretability experiment may identify a mechanism in a particular model. Yet, none of these explain at what point model predictability or reliability falls apart, the durability of computation over longer time horizons, or how local operations produce coherent macroscopic behavior. These limits become consequential when models leave the finite regimes covered by their evaluations (see also the recent examples of misalignment between actual and desired behaviors with OpenAI, Claude Opus 4.6 or Claude Opus 4). We ask them to process longer contexts, maintain persistent state, adapt to users, execute plans over hours or days, and interact with other models. A benchmark measures a finite set of behaviors under a fixed protocol, but it cannot establish which properties persist as reasoning time grows, memory is updated, or several tested components are composed into an untested system. 2. Toward an axiomatics of intelligence Axioms, in a mature science, are chosen to expose invariants. They say what must be true before a theory can speak at all. In mechanics, one asks what is conserved, what is local, what is symmetric, what is stable under limiting procedures. In probability, one asks what it means to compose uncertainty. In computation, one asks what resources are required and what reductions preserve difficulty. We wish for an AI model that gives an answer to the following question: what large-scale behavior follows from the rules by which the system evolves? This is a variant of the program David Hilbert formulated in his sixth problem in 1900. Hilbert called for physical sciences to be treated axiomatically, and singled out the limiting passage from the motion of individual atoms to the laws governing continuous matter. The objective was to prove that microscopic and macroscopic descriptions are two scales of the same system. The central difficulty is choosing microscopic rules that are simple enough to analyze yet expressive enough to support real reasoning. Physics could begin from particles because matter supplied the microscopic substrate. For intelligence, biology provides the analogous starting point: the brain is an existence proof that local communication and adaptation among many simple units can produce coherent cognition. 3. The brain as the source of inspiration for axioms Choosing the brain as the source of inspiration for axioms is a methodological choice. We do not claim that AI must necessarily reproduce the brain. Rather, we believe that the brain remains the strongest evidence that efficient, general intelligence can emerge by establishing proper particle dynamics under severe constraints on energy, locality, and communication. For that reason, the brain is the empirical starting point for our axiomatics. It demonstrates that biological intelligence can arise from sparse, local, adaptive, energy-efficient dynamics on a large evolving graph, without centralized controllers or dense global updates. Neurons fire in brief spikes, synapses adapt using local information, and activity patterns remain highly sparse. In the brain, learning is online and asynchronous: synaptic changes occur continuously during ongoing activity rather than in synchronized training phases, and they integrate evidence across a hierarchy of timescales, from fast sensory responses to slower, more persistent internal states. From this perspective, memory is not a separate module but part of the temporal fabric of computation. Short-lived activity patterns support immediate perception and working memory, while slower synaptic changes and structural adaptations support longer-term expectations and concepts. Predictive processing views emphasize that neurons continuously revise their activity to match anticipated future states, turning the cortex into a distributed model that learns by predicting its own activity over time. Crucially, all of this unfolds in a sparse regime, where at any given moment, only a small fraction of neurons and synapses are active, yet the system maintains a coherent internal model of the world. The underlying graph is immense and highly constrained. Roughly 10 billion neurons are connected by about 10,000 times as many synapses, orders of magnitude fewer than a dense all-to-all connectivity pattern, like in the Transformer, would imply. Indeed, most neurons only interact with a tiny subset of their neighbors, and global behavior arises from many overlapping, locally grounded circuits rather than centralized controllers, like a global road system rather than a direct highway between every pair of homes. The brain thus offers a concrete example of how high-level functions such as reasoning, abstraction, and continual learning can emerge from sparse, local, adaptive, and energy-efficient dynamics on a large evolving graph. 4. Introducing the Equations of Reasoning The Equations of Reasoning are a handful of local rules from which reasoning is meant to emerge, the way theorems follow from axioms. They rewrite attention as a local physical process on a graph of neurons as follows. First, each token activates a set of neurons. Then, their activity reads and updates the memory stored in the synapses, while the learned graph determines where activity flows next. They frame AI reasoning as the process by which memory is used and updated. We can dive deeper into how the Equations of Reasoning define the state-space dynamics of a graph of neurons simulating a learning process similar to the brain’s. The equivalences are formal. Let's start with the nodes (neurons), before defining their interactions at the edge level (synapses). Neurons evolve across time, which we can model with two different timescales: (a) Sequential loops that represent the numerous, time-ordered acquisitions of new beliefs or facts. (b) Each loop is composed of several steps (4 in the BDH paper) where different states represent the evolution of the belief within each acquisition process. Within each acquisition loop, we have: XXX, holding the initial idea of the system at the beginning of the process combined with the new raw input. GxG_xGx​ represents the graph of neurons XiX_iXi​, iii being an index over all the neurons. YYY, that will hold the resulting idea after its confrontation with new (external) input signals. GyG_yGy​ represents the graph of neurons YiY_iYi​. AAA, the accumulator, an intermediate state to help transition from XXX to YYY. The interactions between the different neurons are based on a combination of Hebbian learning, stating that “neurons that fire together wire together”, and modus ponens, or the principle that if XXX implies YYY, then becoming more confident in XXX should also increase confidence in YYY. We introduce the edge notations: GsG_sGs​ is the graph of edges (the memory topology), with fast-weight-like variables σ(i,j)\sigma(i,j)σ(i,j) encoding the curre [truncated for AI cost control]