2026-05-25原文

When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions

Research shows chain-of-thought reasoning is not always beneficial; early entropy dynamics can indicate when reasoning helps. The authors propose EDRM, an adaptive routing framework that uses entropy trajectories, achieving 41-55% token reduction across 15 benchmarks while improving accuracy.

Article intelligence

EngineersAdvanced

Key points

CoT reasoning provides marginal or negative gains on factual and open-ended tasks
Reasoning is a dynamic decoding state signaled by early entropy reduction
EDRM is a training-free routing framework for adaptive inference strategy selection
Selective reasoning outperforms default CoT in efficiency and accuracy

Why it matters

This matters because coT reasoning provides marginal or negative gains on factual and open-ended tasks.

Technical impact

May affect model selection, inference cost, product capability, and evaluation benchmarks.

[2605.22873] When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions

[Submitted on 20 May 2026]

Title:When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions

View a PDF of the paper titled When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions, by Wei Xia and 3 other authors

View PDF HTML (experimental)

Abstract:Chain-of-thought (CoT) reasoning has become the default strategy for enhancing LLM capabilities, yet its application raises a fundamental question: when is explicit reasoning actually beneficial? Empirical evidence reveals a striking paradox: CoT often provides marginal or even negative gains on factual and open-ended tasks while multiplying token consumption. In this work, we show that LLM reasoning is not a static property of tasks or models, but a \emph{dynamic decoding state} that emerges during generation. Through systematic analysis, we find early-stage entropy dynamics provide a reliable signal of this state: tasks benefiting from CoT exhibit consistent entropy reduction, while others display unstable or increasing patterns. This behavior can be interpreted as a phase-transition-like shift from a high-entropy exploratory regime to a low-entropy structured reasoning regime. Based on these insights, we propose \textbf{EDRM} (Entropy Dynamics-based Reasoning Manifold), a lightweight and training-free routing framework that leverages early decoding entropy to adaptively select inference strategies. EDRM embeds entropy trajectories into a compact and interpretable manifold representation, enabling both zero-shot deployment and fine-grained instance-level adaptation. Across 15 benchmarks and 4 LLMs of varying scales and architectures, EDRM consistently outperforms static baselines. At the dataset level, EDRM achieves \textbf{41--55\%} token reduction while improving accuracy with as few as 50 calibration samples. At the instance level, it further improves accuracy by up to \textbf{4.7\%} while maintaining \textbf{27--45\%} token savings. These results suggest that reasoning should be invoked selectively rather than by default, and demonstrate the effectiveness of entropy-driven decoding control for efficient and adaptive LLM inference.

Subjects:

Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

Cite as: arXiv:2605.22873 [cs.LG]

(or arXiv:2605.22873v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2605.22873

arXiv-issued DOI via DataCite

Submission history

From: Wei Xia [view email] [v1] Wed, 20 May 2026 03:15:46 UTC (1,893 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions, by Wei Xia and 3 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.LG

new | recent | 2026-05

Change to browse by:

cs cs.AI cs.CL

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)