Skip to content
AI News HubLIVE
Original source2 min read

MARCH: Scaling Recurrent Memory with Content-Routed State Anchors

Summary

MARCH (Memory-Anchor Routing across Context History) scales recurrent models beyond a fixed-size state by caching state checkpoints as content-addressed 'state anchors.' This gives models a memory bank that grows with context length while preserving efficient recurrent decoding. After standard pretraining, MARCH outperforms several linear attention variants on commonsense reasoning, LongBench, and in-context retrieval.

SourcearXiv Machine LearningAuthor: Ming Zhang, Kaisen Yang, Shu Yu, Ermo Hua, Ning Ding, Xia Hu, Bowen Zhou, Chaochao Lu, Youbang Sun
MARCH: Scaling Recurrent Memory with Content-Routed State Anchors
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[Submitted on 12 Aug 2026]

Title:MARCH: Scaling Recurrent Memory with Content-Routed State Anchors

View a PDF of the paper titled MARCH: Scaling Recurrent Memory with Content-Routed State Anchors, by Ming Zhang and 8 other authors

View PDF HTML (experimental)

Abstract:Transformers owe much of their strong long-context retrieval capability to a token-level memory that grows with context length. This flexibility, however, incurs a quadratic computation complexity during training and a key--value cache that grows linearly during autoregressive inference. Recurrent alternatives offer efficient decoding by compressing the entire history into a fixed-size state, but often underperform on recall-intensive tasks since earlier associations usually get overwritten by subsequent updates, and only the most recent contextual information is retained. In this paper, we introduce Memory-Anchor Routing across Context History (MARCH), a network architecture that effectively scales state-space models beyond a fixed-size dimension, while maintaining computational efficiency over long-sequences. MARCH periodically caches cumulative recurrent-state checkpoints as state anchors and associates each anchor with a compact, content-conditioned anchor key. This lets MARCH maintain a memory bank, which can grow as context length increases, providing a controllable trade-off between historical resolution and memory cost. At each token, MARCH produces an anchor query to attend all causally available state anchors, and the output is calculated as an attention-style aggregation over all historical anchors along the current state. We show that after standard pretraining, MARCH consistently outperforms multiple linear attention variants across commonsense reasoning, LongBench, and in-context retrieval. These results demonstrate that content-routed state caching substantially strengthens recurrent long-range memory while preserving its native computation path.

Subjects:

Machine Learning (cs.LG)

Cite as: arXiv:2608.12435 [cs.LG]

(or arXiv:2608.12435v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2608.12435

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Youbang Sun [view email] [v1] Wed, 12 Aug 2026 13:45:01 UTC (725 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled MARCH: Scaling Recurrent Memory with Content-Routed State Anchors, by Ming Zhang and 8 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.LG

new | recent | 2026-08

Change to browse by:

cs

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

IArxiv recommender toggle

IArxiv Recommender (What is IArxiv?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • MARCH introduces periodic state anchors with content-conditioned keys, giving recurrent models a growing memory bank.
  • It preserves efficient recurrent computation while enabling attention-style retrieval over historical state checkpoints.
  • The method outperforms multiple linear attention variants on reasoning, long-context, and retrieval benchmarks.

Highlights and analysis are generated automatically and may contain errors. Check the original source.

More recent stories on this topic · Models