Skip to content
AI News HubLIVE
Original source2 min read

Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding

Summary

Dustin is a sparse verification framework for long-context speculative decoding that combines draft model lookahead signals with target model historical attention to identify critical tokens, achieving 27.85x self-attention speedup and 9.17x end-to-end decoding speedup at 32k sequence length on Qwen2.5-72B with negligible accuracy loss.

SourcearXiv Computational LinguisticsAuthor: WenHung Lee, Jian-Jia Chen, Xiaolin Lin, Pei-Shuo Wang, Chi-Chih Chang, Chun-Che Yang, Ning-Chi Huang, Grace Li Zhang, Kai-Chiang Wu
Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[2606.24957] Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding

[Submitted on 23 Jun 2026]

Title:Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding

View a PDF of the paper titled Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding, by WenHung Lee and 8 other authors

View PDF HTML (experimental)

Abstract:While speculative decoding improves inference throughput for multi-batch long-context Large Language Models (LLMs), its efficiency is often limited by a verification bottleneck where Key-Value (KV) cache loading dominates latency. Existing compression methods fail in this regime: static eviction incurs accuracy loss due to saliency shift, while dynamic selection introduces prohibitive computational overhead during the verification path. We propose Dustin, a sparse verification framework designed for long-context speculative decoding. Dustin integrates lookahead signals from the draft model with historical attention from the target model to identify critical tokens with high fidelity across multi-step verification windows. To reduce recomputation latency, this approach further employs a sparse estimation scheme that restricts importance scoring to a minimal subset of attention heads. Evaluations on PG-19 and LongBench with Qwen2.5-72B demonstrate that Dustin achieves a 27.85x speedup in self-attention and a 9.17x end-to-end decoding speedup at a 32k sequence length, all with negligible accuracy degradation.

Comments: Accepted to ICML 2026. 9 pages main text, includes references and appendix

Subjects:

Computation and Language (cs.CL); Machine Learning (cs.LG)

Cite as: arXiv:2606.24957 [cs.CL]

(or arXiv:2606.24957v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2606.24957

arXiv-issued DOI via DataCite

Submission history

From: Jian Jia Chen [view email] [v1] Tue, 23 Jun 2026 08:51:20 UTC (25,945 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding, by WenHung Lee and 8 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.CL

new | recent | 2026-06

Change to browse by:

cs cs.LG

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

InvestorsAdvanced

Key points

  • Speculative decoding for long-context LLMs is bottlenecked by KV cache loading during verification
  • Existing compression methods (static eviction or dynamic selection) fail to balance efficiency and accuracy
  • Dustin integrates draft model lookahead and target model historical attention for high-fidelity critical token identification
  • Achieves 27.85x self-attention and 9.17x end-to-end speedup at 32k length with negligible degradation

Highlights and analysis are generated automatically and may contain errors. Check the original source.

More recent stories on this topic · Models