Lingzhe Zhang
Author directory2026
How Much Parallelism Is "Free"? A Principle of Near-Free Parallelism for Parallel Decoding
Minghua He | Lingzhe Zhang | Yuan Liu | Zhou Xiao | Aiwei Liu
Findings of the Association for Computational Linguistics: EMNLP 2026
Minghua He | Lingzhe Zhang | Yuan Liu | Zhou Xiao | Aiwei Liu
Findings of the Association for Computational Linguistics: EMNLP 2026
Parallel decoding improves generation efficiency by processing multiple decode positions within a single decode forward, but reported speedups conflate algorithmic token utilization with the system cost of executing multiple positions. We isolate the system side by introducing Near-Free Parallelism (NFP), the maximum number of positions executable at near-free latency. Analyzing Dense FFNs, MoE FFNs, and Attention against an idle-compute baseline, we find that NFP is shaped not by memory-bound resource slack alone, but also by implementation-induced kernel-granularity slack. Based on these mechanisms, we establish a Near-Free Parallelism principle that predicts the NFP boundary from hardware balance and implementation granularity. Validation on representative Dense and MoE models—spanning both diffusion and autoregressive decoding—shows that the principle accurately predicts practical NFP boundaries, revealing that the standard idle-compute intuition over-predicts them by 2.4–23x in over half of the evaluated deployment settings—offering a system-side budget for parallelism selection and model-system co-design.
d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Models
Leyi Pan | Shuchang Tao | Yunpeng Zhai | Zheyu Fu | Liancheng Fang | Minghua He | Lingzhe Zhang | Zhaoyang Liu | Bolin Ding | Aiwei Liu | Lijie Wen
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Leyi Pan | Shuchang Tao | Yunpeng Zhai | Zheyu Fu | Liancheng Fang | Minghua He | Lingzhe Zhang | Zhaoyang Liu | Bolin Ding | Aiwei Liu | Lijie Wen
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Reinforcement learning (RL) is pivotal for enhancing the reasoning capabilities of diffusion large language models (dLLMs). However, existing dLLM policy optimization methods suffer from two critical reliability bottlenecks: (1) reward sparsity, arising from coarse or unverifiable signals that impede accurate advantage calculation; and (2) their probability estimates do not account for the gap to the unbiased expectation over all decoding orders, which are intractable to compute. To mitigate these issues, we propose d-TreeRPO, a reliable RL framework for dLLMs that leverages tree-structured rollouts and bottom-up advantage computation based on verifiable outcome rewards to provide fine-grained and verifiable step-wise reward signals. Furthermore, we provide a theoretical proof demonstrating that increasing prediction confidence effectively minimizes the gap between unbiased expected prediction probabilities and its single-step forward pass estimate. Guided by this analysis, we introduce a time-scheduled self-distillation loss during training that enhances prediction confidence in later training stages, thereby enabling more accurate probability estimation and better performance. Experiments demonstrate that d-TreeRPO outperforms existing baselines and achieves significant improvements across multiple reasoning benchmarks. Specifically, it achieves +86.2% on Sudoku, +51.6% on Countdown, +4.5% on GSM8K, and +5.3% on Math500 compared to the base model.