Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models

content type paperpublished August 2026

Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models

AuthorsMinseo Kim†, Coleman Hooper‡, Aditya Tomar‡, Chenfeng Xu‡††, Mehrdad Farajtabar, Michael W. Mahoney‡§¶, Kurt Keutzer‡, Amir Gholami‡§*

View publication

Large Language Models (LLMs) have achieved state-of-the-art performance on a broad range of Natural Language Processing (NLP) tasks, including document processing and code generation. Autoregressive Language Models (ARMs), which generate tokens sequentially conditioned on all previous tokens, have been the predominant paradigm for LLMs. While these models have achieved high accuracy across a range of downstream tasks, they exhibit low arithmetic intensity due to the inherent sequential dependency in next-token prediction. Recently, Diffusion Language Models (DLMs) have emerged as a promising alternative architecture. DLMs generate output tokens in parallel, mitigating the limitations of sequential decoding. However, the performance implications of DLMs relative to commonly deployed ARMs are not fully understood. In this work, we present a comprehensive study of the performance characteristics of ARMs and DLMs, combining theoretical analysis with empirical profiling to characterize the trade-offs between these approaches. We show that although DLMs can achieve higher arithmetic intensity than ARMs by leveraging parallelism across token positions, they fail to scale effectively with longer contexts. We then explore block-wise decoding for DLMs, which decouples arithmetic intensity from sequence length and enables better scaling to long contexts (similar to ARMs). We also examine batched inference and find that ARMs exhibit superior throughput as they benefit more from parallelism across sequences in the batch. Finally, we highlight opportunities for accelerating DLM inference, emphasizing that reducing the number of sampling steps is key for open-source DLMs to achieve lower latency relative to ARMs.

† Seoul National University

‡ University of California, Berkeley

§ ICSI

¶ LBNL

†† University of Texas at Austin

• Advisory role

Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential

August 8, 2025research area Methods and Algorithms, research area Speech and Natural Language Processing

Autoregressive language models are constrained by their inherently sequential nature, generating one token at a time. This paradigm limits inference speed and parallelism, especially during later stages of generation when the direction and semantics of text are relatively certain. In this work, we propose a novel framework that leverages the inherent knowledge of vanilla autoregressive language models about future tokens, combining techniques to…

Read more

Scaling Diffusion Language Models via Adaptation from Autoregressive Models

April 16, 2025research area Speech and Natural Language Processingconference ICLR

Diffusion Language Models (DLMs) have emerged as a promising new paradigm for text generative modeling, potentially addressing limitations of autoregressive (AR) models. However, current DLMs have been studied at a smaller scale compared to their AR counterparts and lack fair comparison on language modeling benchmarks. Additionally, training diffusion models from scratch at scale remains challenging. Given the prevalence of open-source AR…

Read more