2026-05-11原文

More Thinking, More Bias: Length-Driven Position Bias in Reasoning Models

This paper challenges the assumption that chain-of-thought reasoning reduces bias, demonstrating that position bias in multiple-choice QA actually increases with reasoning trajectory length. Across 13 configurations, 12 show a positive partial correlation between trajectory length and Position Bias Score (PBS). Truncation experiments confirm causality, and the 671B DeepSeek-R1 shows low overall bias but a persistent length effect in the longest quartile. Direct-answer position bias is a distinct phenomenon. The findings argue against assuming reasoning models are order-robust and provide a diagnostic toolkit.

Article intelligence

InvestorsAdvanced

Key points

Position bias scales with reasoning trajectory length across multiple reasoning-capable models, even after controlling for accuracy.
Truncation intervention causally links longer reasoning to increased bias toward position-preferred options (16% to 32% for R1-Qwen-7B).
DeepSeek-R1 (671B) exhibits near-zero overall PBS but significant bias in the longest trajectory quartile (PBS=0.071).
Direct-answer position bias and length-driven bias are distinct phenomena with different footprints.

Why it matters

This matters because position bias scales with reasoning trajectory length across multiple reasoning-capable models, even after controlling for accuracy.

Technical impact

May affect model selection, inference cost, product capability, and evaluation benchmarks.

[2605.06672] More Thinking, More Bias: Length-Driven Position Bias in Reasoning Models

[Submitted on 21 Apr 2026]

Title:More Thinking, More Bias: Length-Driven Position Bias in Reasoning Models

View a PDF of the paper titled More Thinking, More Bias: Length-Driven Position Bias in Reasoning Models, by Xiao Wang

View PDF HTML (experimental)

Abstract:Chain-of-thought (CoT) reasoning and reasoning-tuned models such as DeepSeek-R1 are commonly assumed to reduce shallow heuristic biases by thinking carefully. We test this on position bias in multiple-choice QA and find a different story: within any reasoning-capable model, per-question position bias scales with the length of the reasoning trajectory.

Across thirteen reasoning-mode configurations (two R1-distilled 7-8B models, two base models prompted with CoT, and DeepSeek-R1 at 671B) on MMLU, ARC-Challenge, and GPQA, twelve show a positive partial correlation between trajectory length and Position Bias Score (PBS) after controlling for accuracy, ranging from 0.11 to 0.41 (all p

new | recent | 2026-05

Change to browse by:

cs cs.CL cs.LG

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)