OpenNovelty.org

Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding

ICLR 2026 Conference SubmissionAnonymous Authors
OpenReview Score: 7.0Download Report PDF
Diffusion LLMefficiency
Abstract:

Diffusion-based large language models (Diffusion LLMs) have shown promise for non-autoregressive text generation. However, the practical inference speed of open-sourced Diffusion LLMs often lags behind autoregressive models due to the lack of Key-Value (KV) Cache and quality degradation when decoding multiple tokens simultaneously. To bridge this gap, we introduce Fast-dLLM, a method that incorporates a novel block-wise approximate KV Cache mechanism tailored for bidirectional diffusion models, enabling cache reuse with negligible performance drop. Additionally, we identify the root cause of generation quality degradation in parallel decoding as the disruption of token dependencies under the conditional independence assumption. To address this, Fast-dLLM also proposes a confidence-aware parallel decoding strategy that selectively decodes tokens exceeding a confidence threshold, mitigating dependency violations and maintaining generation quality. Experimental results on LLaDA and Dream models across multiple LLM benchmarks demonstrate up to 27.6× throughput improvement with minimal accuracy loss, closing the performance gap with autoregressive models and paving the way for practical deployment of Diffusion LLMs.

Disclaimer
This report is AI-GENERATED using Large Language Models and WisPaper (A scholar search engine). It analyzes academic papers' tasks and contributions against retrieved prior work. While this system identifies POTENTIAL overlaps and novel directions, ITS COVERAGE IS NOT EXHAUSTIVE AND JUDGMENTS ARE APPROXIMATE. These results are intended to assist human reviewers and SHOULD NOT be relied upon as a definitive verdict on novelty.
NOTE that some papers exist in multiple, slightly different versions (e.g., with different titles or URLs). The system may retrieve several versions of the same underlying work. The current automated pipeline does not reliably align or distinguish these cases, so human reviewers will need to disambiguate them manually.
If you have any questions, please contact: mingzhang23@m.fudan.edu.cn

Overview

Overall Novelty Assessment

The paper proposes Fast-dLLM, which combines a block-wise approximate KV cache mechanism with a confidence-aware parallel decoding strategy to accelerate diffusion-based large language models. According to the taxonomy, it resides in the 'Confidence-Based Token Selection' leaf under 'Decoding Strategy Optimization', alongside two sibling papers. This leaf represents a moderately populated research direction within a broader taxonomy of 50 papers across approximately 36 topics, suggesting that confidence-based approaches are an established but not overcrowded area of investigation in diffusion LLM acceleration.

The taxonomy reveals that Fast-dLLM sits at the intersection of two major acceleration paradigms: 'Decoding Strategy Optimization' (which includes adaptive parallel decoding and planning-based methods) and 'Cache-Based Acceleration' (covering adaptive and block-wise KV cache techniques). Neighboring leaves include 'Adaptive Parallel Decoding' and 'Block-Wise KV Cache', indicating that the paper bridges token selection strategies with caching mechanisms. The taxonomy's scope notes clarify that confidence-based methods focus on model confidence scores for token unmasking, distinguishing them from planning-based trajectory optimization or purely architectural modifications.

Among 28 candidates examined through limited semantic search, the analysis identified 11 refutable pairs across three contributions. The block-wise KV cache mechanism examined 10 candidates with 7 appearing to provide overlapping prior work, suggesting substantial existing research on caching for diffusion models. The confidence-aware parallel decoding strategy examined 9 candidates with only 2 refutable matches, indicating potentially greater novelty in this specific combination. The overall Fast-dLLM framework also examined 9 candidates with 2 refutable pairs, though the limited search scope means these statistics reflect top-K semantic matches rather than exhaustive coverage.

Based on the limited literature search of 28 candidates, the work appears to synthesize existing acceleration paradigms—caching and confidence-based decoding—in a novel combination tailored for diffusion LLMs. The higher refutation rate for the caching component suggests this aspect builds more directly on established techniques, while the confidence-aware strategy may represent a less explored integration. The analysis does not cover the full breadth of diffusion LLM research, and a more comprehensive search might reveal additional overlapping work in either component.

Taxonomy

Core-task Taxonomy Papers
50
3
Claimed Contributions
28
Contribution Candidate Papers Compared
11
Refutable Paper

Research Landscape Overview

Core task: Accelerating inference of diffusion-based large language models. The field has organized itself around several complementary acceleration strategies. Decoding Strategy Optimization explores how to intelligently select or schedule tokens during the iterative diffusion process, including confidence-based selection and adaptive scheduling approaches. Cache-Based Acceleration (e.g., dLLM-Cache[10], dCache[26]) focuses on reusing intermediate computations across diffusion steps to reduce redundant calculations. Architectural Acceleration and Model Compression branches address efficiency through structural modifications and quantization techniques (DLLMQuant[23]), while Speculative and Hybrid Decoding methods (Diffuspec[14], Self Speculative Decoding[12]) attempt to predict multiple tokens or steps ahead to reduce sequential dependencies. Training and Objective Optimization investigates how learning procedures can be redesigned for faster convergence, and foundational branches cover theoretical underpinnings (Convergence Theory[22], Discrete Diffusion Models[29]) alongside multimodal extensions and application-specific adaptations. Survey works (Diffusion Language Survey[5], Diffusion LLM Survey[28]) provide broader perspectives on these evolving directions. Particularly active lines of work center on reducing the number of diffusion steps required and exploiting token-level confidence to skip unnecessary computations. Fast-dLLM[0] sits within the Confidence-Based Token Selection cluster, emphasizing early stopping or selective refinement of high-confidence tokens to avoid redundant denoising iterations. This approach contrasts with neighboring methods like Saber[19], which may prioritize different scheduling heuristics, and Self Speculative Decoding[12], which leverages draft-and-verify mechanisms rather than confidence thresholds. The trade-off across these branches often involves balancing generation quality against wall-clock speedup: confidence-based strategies can yield substantial gains when token predictions stabilize quickly, but may require careful tuning to avoid premature convergence. Fast-dLLM[0] exemplifies this balance by targeting scenarios where diffusion models exhibit predictable confidence dynamics, positioning it as a practical complement to cache-based and speculative techniques that address orthogonal bottlenecks in the inference pipeline.

Claimed Contributions

Block-wise approximate KV Cache mechanism for bidirectional diffusion models

The authors propose a novel KV caching strategy tailored for masked diffusion language models that use full bidirectional attention. By adopting block-wise generation and caching prefix (and optionally suffix) tokens, the method enables substantial computational reuse across decoding steps with negligible performance degradation.

10 retrieved papers
Can Refute
Confidence-aware parallel decoding strategy

The authors introduce a dynamic decoding approach that selectively decodes tokens based on confidence thresholds rather than a fixed count per step. This strategy mitigates token-dependency violations under the conditional independence assumption and maintains generation quality while accelerating inference by up to 13.3×.

9 retrieved papers
Can Refute
Fast-dLLM framework achieving state-of-the-art acceleration for Diffusion LLMs

The authors present Fast-dLLM, an integrated framework combining block-wise KV caching and confidence-aware parallel decoding. Experiments show up to 27.6× end-to-end speedup on multiple benchmarks with minimal accuracy loss, closing the performance gap with autoregressive models and enabling practical deployment of Diffusion LLMs.

9 retrieved papers
Can Refute

Core Task Comparisons

Comparisons with papers in the same taxonomy category

Contribution Analysis

Detailed comparisons for each claimed contribution

Contribution

Block-wise approximate KV Cache mechanism for bidirectional diffusion models

The authors propose a novel KV caching strategy tailored for masked diffusion language models that use full bidirectional attention. By adopting block-wise generation and caching prefix (and optionally suffix) tokens, the method enables substantial computational reuse across decoding steps with negligible performance degradation.

Contribution

Confidence-aware parallel decoding strategy

The authors introduce a dynamic decoding approach that selectively decodes tokens based on confidence thresholds rather than a fixed count per step. This strategy mitigates token-dependency violations under the conditional independence assumption and maintains generation quality while accelerating inference by up to 13.3×.

Contribution

Fast-dLLM framework achieving state-of-the-art acceleration for Diffusion LLMs

The authors present Fast-dLLM, an integrated framework combining block-wise KV caching and confidence-aware parallel decoding. Experiments show up to 27.6× end-to-end speedup on multiple benchmarks with minimal accuracy loss, closing the performance gap with autoregressive models and enabling practical deployment of Diffusion LLMs.

Table of Contents