2026-07-17 04:00 UTCOriginal source2 min readUpdated: 2026-07-17 08:25 UTC

Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs

Polestar is a training-free inference framework that addresses KV-cache reuse and decoding parallelism challenges in diffusion LLMs by leveraging token representation drift. It consists of Polestar-Cache for sparse cache refreshes and Polestar-Commit for identifying commit-ready tokens, achieving up to 10.73% accuracy improvement and 3.7x higher throughput on math and coding benchmarks.

SourcearXiv Computational LinguisticsAuthor: Mingyu Lee, Akshat Ramachandran, Souvik Kundu, Tushar Krishna

-->

[Submitted on 7 May 2026]

Title:Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs

View a PDF of the paper titled Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs, by Mingyu Lee and 3 other authors

View PDF HTML (experimental)

Abstract:The inference efficiency of diffusion large language models (dLLMs) is constrained by two challenges: bidirectional attention precludes efficient KV-cache reuse, while increasing decoding parallelism with static confidence thresholds can compromise generation quality. We observe that both challenges arise from a shared phenomenon: as tokens are decoded, their contextual integration through bidirectional attention causes token representations to drift (evolve) across decoding steps. This insight motivates Polestar, a training-free inference framework that uses token representation drift as a unified signal to jointly address both challenges. Polestar comprises two components: Polestar-Cache, which identifies stale KV-cache positions via drift and performs sparse KV-cache refreshes to enable efficient reuse, and Polestar-Commit, which detects sharp drift events to reliably identify commit-ready tokens. Across mathematics and coding benchmarks on several dLLM families, Polestar sets a new state of the art on the accuracy-throughput Pareto frontier, achieving up to 10.73% accuracy improvement, up to 3.7x higher throughput, and high decoding parallelism of 3.67 tokens per forward pass over existing baselines.

Subjects:

Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Cite as: arXiv:2607.14107 [cs.CL]

(or arXiv:2607.14107v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2607.14107

arXiv-issued DOI via DataCite

Submission history

From: Akshat Ramachandran [view email] [v1] Thu, 7 May 2026 18:05:39 UTC (3,600 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs, by Mingyu Lee and 3 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.CL

new | recent | 2026-07

Change to browse by:

cs cs.AI

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)