Skip to content
AI News HubLIVE
Original source2 min read

Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors

Summary

A paper accepted at the ICML 2026 EIML workshop proposes "stable miscalibration": confident wrong answers that remain locally stable under small perturbations, rather than reflecting fragile inference. Combining an output-level audit score with an internal hidden-state sensitivity probe, the author finds that self-critical prompting reduces layer-wise sensitivity in three open-weight models, but stresses that such stabilization does not equal calibration.

SourcearXiv AIAuthor: Akira Okutomi
Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[Submitted on 15 Jul 2026]

Title:Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors

View a PDF of the paper titled Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors, by Akira Okutomi

View PDF HTML (experimental)

Abstract:High-confidence errors in large language models are often treated as evidence of fragile internal inference. We study a different possibility: stable miscalibration, where a confident wrong answer remains locally stable under small perturbations. We combine two diagnostics: a label-aware output-level audit score that ranks domains by confidence variation and overconfident mistakes under a forced-answer baseline, and an internal sensitivity probe that measures hidden-state movement. On a multi-domain binary factual audit set, this audit score tracks where abstention-aware self-critique reduces decision loss, although direct labeled baselines rank the same gain more strongly. Internally, self-critical prompting consistently reduces hidden-state sensitivity across layers in three open-weight models. This supports prompt-induced local stabilization rather than a purely output-level abstention pattern, but it does not imply calibration: audit-defined overconfident errors are not clearly more locally sensitive than confidently correct answers, so some high-confidence errors may be stable and miscalibrated rather than simply fragile.

Comments: Accepted at the 2nd Workshop on Epistemic Intelligence in Machine Learning (EIML)@ICML 2026. 7 pages, 4 figures

Subjects:

Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

Cite as: arXiv:2608.13591 [cs.AI]

(or arXiv:2608.13591v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2608.13591

arXiv-issued DOI via DataCite

Submission history

From: Akira Okutomi [view email] [v1] Wed, 15 Jul 2026 06:27:17 UTC (1,204 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors, by Akira Okutomi

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.AI

new | recent | 2026-08

Change to browse by:

cs cs.CL

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • Introduces "stable miscalibration," arguing some high-confidence errors are locally stable under perturbation rather than signs of fragile reasoning.
  • Combines a label-aware output-level audit score with an internal hidden-state sensitivity probe as a two-level diagnostic.
  • Self-critical prompting consistently reduces hidden-state sensitivity across layers in three open-weight models.
  • Stabilization is not calibration: audit-defined overconfident errors are not clearly more locally sensitive than confidently correct answers.

Highlights and analysis are generated automatically and may contain errors. Check the original source.