2026-06-03原文2 min readUpdated: 2026-06-03

Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States

A study probing Qwen3-14B hidden states shows that linear probes achieving 100% accuracy in classifying reasoning types (deductive, inductive, abductive) actually detect task format confounds (source, option count, response length) rather than genuine reasoning modes. After deconfounding, accuracy drops to chance, and causal steering shows no functional link. The findings urge routine format deconfounding in mechanistic interpretability.

SourcearXiv Computational LinguisticsAuthor: Subramanyam Sahoo, Vinija Jain, Aman Chadha, Divya Chaudhary

[2606.02907] Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States

[Submitted on 1 Jun 2026]

Title:Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States

View a PDF of the paper titled Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States, by Subramanyam Sahoo and 3 other authors

View PDF HTML (experimental)

Abstract:Linear probing of large language model (LLM) hidden states is widely used to claim that models learn distinct representations for different reasoning types. We test this by probing Qwen3-14B on three benchmarks spanning the classical trichotomy: LogiQA 2.0 (deductive), ARC-Challenge (inductive), and $\alpha$NLI (abductive). At layer 32 of 40, linear probes achieve 100\% cross-validated accuracy with well-separated geometry (intrinsic dimensionalities: 20.6, 28.5, 33.6; convex hull contamination $\leq$1.5\%). However, this separation is entirely driven by format confounds. Residualizing source identity, option count, and response length reduces accuracy to chance. Trace-anchor similarity indicates largely shared reasoning across tasks (42.5\% agreement vs.\ 33.3\% chance), and causal steering with random controls ($n=20$) shows no functional link between geometry and reasoning mode ($p=0.286$). Thus, high probe accuracy reflects task format rather than computational structure, motivating routine format deconfounding in mechanistic interpretability.

Comments: Accepted in the 6th Workshop on Trustworthy NLP, ACL 2026

Subjects:

Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Cite as: arXiv:2606.02907 [cs.CL]

(or arXiv:2606.02907v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2606.02907

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Subramanyam Sahoo [view email] [v1] Mon, 1 Jun 2026 21:22:15 UTC (199 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States, by Subramanyam Sahoo and 3 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.CL

new | recent | 2026-06

Change to browse by:

cs cs.AI

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)