Shuang Cheng
Author directory2026
SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation
Shuang Cheng | Yihan Bian | Dawei Liu | Yuhua Jiang | Yihao Liu | Linfeng Zhang | Qian Yao | Zhongbo Tian | Wenhai Wang | Qipeng Guo | Kai Chen | Biqing Qi | Bowen Zhou
Findings of the Association for Computational Linguistics: ACL 2026
Shuang Cheng | Yihan Bian | Dawei Liu | Yuhua Jiang | Yihao Liu | Linfeng Zhang | Qian Yao | Zhongbo Tian | Wenhai Wang | Qipeng Guo | Kai Chen | Biqing Qi | Bowen Zhou
Findings of the Association for Computational Linguistics: ACL 2026
Autoregressive (AR) language modeling remains the dominant paradigm due to its dense supervision signal and highly optimized serving infrastructure, but its strictly causal, token-by-token decoding limits parallelism and non-causal modeling. While masked diffusion offers a promising path toward parallel generation, it faces two critical bottlenecks: training inefficiency stemming from sparse masked objectives, and high latency caused by iterative whole-sequence denoising. We present a systematic study of blockwise discrete diffusion, a pragmatic middle ground that preserves AR-compatible serving while enabling parallel intra-block generation. Our study proceeds in four steps: (i) a controlled, compute- and scale-matched comparison revealing that AR is a more effective backbone for blockwise hybrids than masked diffusion objectives; (ii) a scalable conversion recipe, SDAR, validating that AR models spanning 1.7B to 30B parameters can be adapted into block diffusion models with minimal compute while preserving backbone capabilities; and (iii) a systematic characterization of decoding dynamics, which reveals a virtuous cycle where larger models enable more aggressive parallel decoding, achieving theoretical speedups over 5× and wall-clock speedups of 2.3× on H200 GPUs in latency-critical regimes; and (iv) an investigation of local non-causal modeling capabilities, showing that SDAR’s local bidirectional attention overcomes causal bottlenecks in scientific domains (e.g., chemistry) and enables robust test-time scaling. We release the full model suite, the training framework, and our inference engines for further innovation in non-autoregressive generative paradigms.
MacAgentBench: Benchmarking AI Agents on Real-World macOS Desktop
Yikun Fu | Bowen Fu | Zhenyu Wu | Shuang Cheng | Xiaowei Sun | Bowen Yang | Zehao Li | Yibo Zhao | Zichen Ding | Zhoumianze Liu | Shijie Wang | Biqing Qi | Bowen Zhou
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yikun Fu | Bowen Fu | Zhenyu Wu | Shuang Cheng | Xiaowei Sun | Bowen Yang | Zehao Li | Yibo Zhao | Zichen Ding | Zhoumianze Liu | Shijie Wang | Biqing Qi | Bowen Zhou
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Computer use agents (CUAs) have advanced rapidly in desktop automation, and a growing number of users deploy CUAs such as OpenClaw on Mac Mini for always-on automation. However, existing benchmarks, including those for macOS, evaluate agents without framework augmentation and rely on binary evaluation. As a result, they fail to capture both the framework capabilities leveraged by modern CUAs and the partial progress on long-horizon, multi-application tasks. We present MacAgentBench, a comprehensive macOS agent benchmark comprising 169 seed tasks expanded into 676 task instances across 25 applications, with nearly 60% involving both GUI and CLI interaction. The benchmark adopts deterministic rule-based evaluation and introduces fine-grained multi-checkpoint scoring with capability annotations for multi-application tasks. Experiments across three frameworks and 16 models show that the best configuration, Claude Opus 4.6 on OpenClaw, attains 73.7% Pass@1, while this advantage is primarily driven by the skill library rather than by framework design. Fine-grained metrics further reveal that models with similar Pass@1 can differ substantially in sub-goal completion.
Nirvana: A Specialized Generalist Model With Task-Aware Memory Mechanism
Yuhua Jiang | Shuang Cheng | Yihao Liu | Ermo Hua | Che Jiang | Weigao Sun | Yu Cheng | Feifei Gao | Biqing Qi | Bowen Zhou
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yuhua Jiang | Shuang Cheng | Yihao Liu | Ermo Hua | Che Jiang | Weigao Sun | Yu Cheng | Feifei Gao | Biqing Qi | Bowen Zhou
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Large Language Models (LLMs) excel at general language tasks but struggle in specialized domains. Specialized Generalist Models (SGMs) address this by preserving broad capabilities while adapting to target domains. However, existing architectures provide limited support for task-guided specialized memory mechanisms. In this work, we introduce Nirvana, an SGM featuring specialized memory, linear-time complexity, and test-time task information extraction. Central to Nirvana are: (1) Task-Aware Memory Trigger (Trigger), which treats each input as a self-supervised fine-tuning task and adjusts task-related parameters on the fly; and (2) Specialized Memory Updater (Updater), which dynamically consolidates task-relevant context. Nirvana matches or surpasses LLM baselines on general benchmarks and achieves the lowest perplexity across specialized domains including biomedicine, finance, and law. On the challenging task of Magnetic Resonance Imaging (MRI), we attach lightweight codecs to the frozen Nirvana backbone and fine-tune them on paired k-space signals and images. Nirvana achieves higher-fidelity reconstructions than conventional LLM-based models, with Trigger providing effective domain-specific adaptation. Ablation studies confirm that removing Trigger leads to substantial degradation across all tasks, underscoring its essential role in task-aware specialization. Models are available at https://huggingface.co/collections/YuhuaJiang/nirvana. Code is available at https://github.com/YuhuaJiang2002/Nirvana.
SDAR-VL: Stable and Efficient Block-wise Diffusion for Vision-Language Understanding
Shuang Cheng | Yuhua Jiang | Zineng Zhou | Dawei Liu | Tao Wang | Linfeng Zhang | Biqing Qi | Bowen Zhou
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Shuang Cheng | Yuhua Jiang | Zineng Zhou | Dawei Liu | Tao Wang | Linfeng Zhang | Biqing Qi | Bowen Zhou
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Block-wise discrete diffusion offers an attractive balance between parallel generation and causal dependency modeling, making it a promising backbone for vision-language modeling. However, its practical adoption has been limited by high training cost, slow convergence, and instability, which have so far kept it behind strong autoregressive (AR) baselines. We present SDAR-VL, the first systematic application of block-wise discrete diffusion to large-scale vision-language understanding (VLU), together with an integrated framework for efficient and stable training. This framework unifies three components: 1) Asynchronous Block-wise Noise Scheduling to diversify supervision within each batch; 2) Effective Mask Ratio Scaling for unbiased loss normalization under stochastic masking; and 3) a Progressive Beta Noise Curriculum that increases effective mask coverage while preserving corruption diversity. Experiments on 21 single-image, multi-image, and video benchmarks show that SDAR-VL consistently improves training efficiency, convergence stability, and task performance over conventional block diffusion. On this evaluation suite, SDAR-VL sets a new state of the art among diffusion-based vision-language models and, under matched settings, matches or surpasses strong AR baselines such as LLaVA-OneVision as well as the global diffusion baseline LLaDA-V, establishing block-wise diffusion as a practical backbone for VLU.
2024
Unlocking Continual Learning Abilities in Language Models
Wenyu Du | Shuang Cheng | Tongxu Luo | Zihan Qiu | Zeyu Huang | Ka Chun Cheung | Reynold Cheng | Jie Fu
Findings of the Association for Computational Linguistics: EMNLP 2024
Wenyu Du | Shuang Cheng | Tongxu Luo | Zihan Qiu | Zeyu Huang | Ka Chun Cheung | Reynold Cheng | Jie Fu
Findings of the Association for Computational Linguistics: EMNLP 2024
2023
Enhancing Multilingual Document-Grounded Dialogue Using Cascaded Prompt-Based Post-Training Models
Jun Liu | Shuang Cheng | Zineng Zhou | Yang Gu | Jian Ye | Haiyong Luo
Proceedings of the Third DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering
Jun Liu | Shuang Cheng | Zineng Zhou | Yang Gu | Jian Ye | Haiyong Luo
Proceedings of the Third DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering
The Dialdoc23 shared task presents a Multilingual Document-Grounded Dialogue Systems (MDGDS) challenge, where system responses are generated in multiple languages using user’s queries, historical dialogue records and relevant passages. A major challenge for this task is the limited training data available in low-resource languages such as French and Vietnamese. In this paper, we propose Cascaded Prompt-based Post-training Models, dividing the task into three subtasks: Retrieval, Reranking and Generation. We conduct post-training on high-resource language such as English and Chinese to enhance performance of low-resource languages by using the similarities of languages. Additionally, we utilize the prompt method to activate model’s ability on diverse languages within the dialogue domain and explore which prompt is a good prompt. Our comprehensive experiments demonstrate the effectiveness of our proposed methods, which achieved the first place on the leaderboard with a total score of 215.40 in token-level F1, SacreBleu, and Rouge-L metrics.
Search
Fix author
Co-authors
- Biqing Qi 4
- Bowen Zhou 4
- Yuhua Jiang 3
- Dawei Liu 2
- Yihao Liu 2
- Linfeng Zhang 2
- Zineng Zhou 2
- Yihan Bian 1
- Kai Chen 1
- Reynold Cheng 1
- Yu Cheng 1
- Ka Chun Cheung 1
- Zichen Ding 1
- Wenyu Du 1
- Bowen Fu 1
- Jie Fu 1
- Yikun Fu 1
- Feifei Gao 1
- Yang Gu 1
- Qipeng Guo 1
- Ermo Hua 1
- Zeyu Huang 1
- Che Jiang 1
- Zehao Li 1
- Jun Liu 1
- Zhoumianze Liu 1
- Haiyong Luo 1
- Tongxu Luo 1
- Zihan Qiu 1
- Weigao Sun 1
- Xiaowei Sun 1
- Zhongbo Tian 1
- Shijie Wang 1
- Tao Wang 1
- Wenhai Wang 1
- Zhenyu Wu 1
- Bowen Yang 1
- Qian Yao 1
- Jian Ye 1
- Yibo Zhao 1