Skip to content
AI News HubLIVE
More
Summary only

A Geometric Calculator Inside a Neural Network

Researchers discover a general-purpose addition module in Llama 3.1 8B that uses circular representations (Fourier features) to encode numbers and perform addition, reused across tasks like month and day arithmetic. The module resides in layer 18, computes modular sums in parallel, and causal steering confirms its role.

SourceHacker News AIAuthor: guybedo
A Geometric Calculator Inside a Neural Network
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions

Summary only

Under the source content permission, this page provides an editorial summary, key points, and a link to the original article.

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • A general-purpose addition module was found in layer 18 of Llama 3.1 8B.
  • Numbers are represented using circles (Fourier features) corresponding to modular residues.
  • The module computes sums for each modulus in parallel and is reused across tasks like month and day addition.
  • Steering experiments causally confirm the module's function.

Highlights and analysis are generated automatically and may contain errors. Check the original source.