Skip to content
AI News HubLIVE
Original source2 min read

Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents

Summary

A new arXiv paper tests 4-bit post-training quantization on multi-turn tool-calling agents and finds that standard task reward stays flat while quantization amplifies pre-existing failure modes—such as tool-name hallucination—by up to 2.5×, with the benchmark's ten-error budget absorbing the extra failures. Shrinking the budget to two errors re-exposes a 17-point score gap in the cell where quantization added error volume.

SourcearXiv Machine LearningAuthor: Jiwon Jang, Kisu Yang, Heuiseok Lim, Hyunwoo Park
Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[Submitted on 29 Jul 2026]

Title:Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents

View a PDF of the paper titled Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents, by Jiwon Jang and 3 other authors

View PDF HTML (experimental)

Abstract:Post-training quantization to 4-bit weights is widely reported to be nearly lossless. We test this claim for multi-turn, tool-calling agents, where it now matters most. On $\tau^2$-bench, across two open-weight model families in dense and MoE variants and two domains (eight cells, 456 episodes each, at 16-, 8-, and 4-bit weights), quantization indeed looks free on the standard metric. No cell shows a score change that survives multiple-comparison correction, and in the cell that carries the largest process damage, equivalence testing bounds the change within $\pm$7.5 points. The process tells a different story. Quantization amplifies the failure the model already exhibits at full precision (tool-name hallucination in telecom, with the same directional trend in retail entity errors) by up to 2.5$\times$ in volume (+17.6 points per task), while creating essentially no new failures. The failure set is the same at every precision (rank correlation $\geq$ 0.94, 0.18% novel events). The score stays flat because the benchmark's ten-error budget absorbs the extra failures. Shrinking the budget to two errors re-exposes a score gap of 17 points, and it does so only in the one cell where quantization added error volume, exactly as the masking account predicts. A targeted error-repair prompt, run for five telecom models at every precision, removes the damage exactly and only where it lives. Both diagnostics, the per-channel error rate and success under a shrinking budget, come from logs benchmarks already collect; we suggest reporting them alongside task reward.

Comments: preprint

Subjects:

Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Cite as: arXiv:2607.27275 [cs.LG]

(or arXiv:2607.27275v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2607.27275

arXiv-issued DOI via DataCite

Submission history

From: Kisu Yang [view email] [v1] Wed, 29 Jul 2026 12:40:04 UTC (131 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents, by Jiwon Jang and 3 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.LG

new | recent | 2026-07

Change to browse by:

cs cs.AI

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

IArxiv recommender toggle

IArxiv Recommender (What is IArxiv?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • Across eight τ²-bench cells, 4-bit quantization shows no statistically significant change in standard task reward, but process metrics reveal up to 2.5× more failures.
  • Quantization creates essentially no new failure types; it amplifies failures already present at full precision (rank correlation ≥ 0.94, only 0.18% novel events).
  • Reducing the error budget from ten to two errors reveals a 17-point score gap only in the cell where quantization increased error volume, confirming the masking effect.
  • The authors recommend reporting per-channel error rate and success under a shrinking budget alongside task reward, using logs benchmarks already collect.

Highlights and analysis are generated automatically and may contain errors. Check the original source.