Shen Nie
Author directory2026
LLaDA-Rec: Discrete Diffusion for Parallel Semantic ID Generation in Generative Recommendation
Teng Shi | Chenglei Shen | Weijie Yu | Shen Nie | Chongxuan Li | Xiao Zhang | Ming He | Jianping Fan | Jun Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Teng Shi | Chenglei Shen | Weijie Yu | Shen Nie | Chongxuan Li | Xiao Zhang | Ming He | Jianping Fan | Jun Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Generative recommendation represents items as semantic identifiers (SIDs), i.e., sequences of discrete tokens, and typically predict the next item autoregressively. Despite their strong performance, such methods suffer from two limitations: (1) unidirectional constraints, where causal attention enforces left-to-right dependencies and limits global semantic modeling; and (2) error accumulation, where early mistakes propagate to later tokens and hurt recommendation accuracy. To address these issues, we propose LLaDA-Rec, a discrete diffusion framework that reformulates next-item recommendation as parallel SID generation. By combining bidirectional attention with an adaptive generation order, LLaDA-Rec captures richer inter-item and intra-item dependencies while reducing error accumulation. Our framework includes three key components: (1) a parallel tokenization mechanism for diffusion-compatible SIDs; (2) masking strategies at both the user-history and next-item levels; and (3) an adapted beam search algorithm for adaptive-order discrete diffusion decoding. Experiments on three public datasets show that LLaDA-Rec consistently outperforms both conventional ID-based recommenders and strong generative baselines, highlighting discrete diffusion as a promising paradigm for generative recommendation.
LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models
Fengqi Zhu | Rongzhen Wang | Shen Nie | Xiaolu Zhang | Chunwei Wu | Jun Zhou | Yankai Lin | Ji-Rong Wen | Chongxuan Li
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Fengqi Zhu | Rongzhen Wang | Shen Nie | Xiaolu Zhang | Chunwei Wu | Jun Zhou | Yankai Lin | Ji-Rong Wen | Chongxuan Li
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Masked diffusion language models present a promising paradigm for language modeling, yet the systematic theoretical analysis and comprehensive empirical validation of their alignment on general tasks remain relatively underexplored. In this paper, we identify the primary challenge for this problem: the high variance in Evidence Lower Bound (ELBO)-based likelihood estimates required for preference optimization. To address this issue, we propose Variance-Reduced Preference Optimization (VRPO), a framework that formally analyzes the bias and variance of the preference optimization loss and gradient based on Direct Preference Optimization, showing both are governed by a score-estimator variance. Building on this foundation, we introduce multiple unbiased variance reduction strategies, including optimal budget allocation and antithetic sampling, to improve alignment performance. We demonstrate the effectiveness of VRPO by applying it to LLaDA, a large diffusion language model. The resulting model, LLaDA 1.5, consistently outperforms its SFT-only predecessor consistently across various general benchmarks, such as mathematics (GSM8K +4.7), coding (HumanEval +3.0, MBPP +1.8), and alignment (IFEval +4.0, Arena-Hard +4.3). Furthermore, LLaDA 1.5 demonstrates a highly competitive mathematical performance compared to other strong language MDMs and ARMs. Our model is available at https://huggingface.co/GSAI-ML/LLaDA-1.5.