Deyang Kong
Author directory2026
Adaptive Test-Time Compute Allocation for Block Diffusion Language Models in Complex Reasoning
Yi Lu | Deyang Kong | Jianing Wang | Linsen Guo | Xue Wang | Qi Guo | Tao Gui | Xuanjing Huang | Wei Ye | Shikun Zhang | Wei Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
Yi Lu | Deyang Kong | Jianing Wang | Linsen Guo | Xue Wang | Qi Guo | Tao Gui | Xuanjing Huang | Wei Ye | Shikun Zhang | Wei Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
Recent advances in block diffusion language models have demonstrated competitive performance and strong scalability on reasoning tasks. However, their test-time compute allocation remains largely unexplored, leaving a critical speed-effectiveness trade-off unresolved in long Chain-of-Thought reasoning. To address this, we propose a unified test-time compute allocation framework that introduces adaptivity in both step-wise decoding and blockwise generation. At the decoding level, we propose Bounded Adaptive Confidence Decoding (BACD), a difficulty-aware sampling strategy that dynamically adjusts denoising based on model confidence, accelerating inference while controlling error accumulation. Beyond step-wise adaptivity, we introduce the Think Coarse, Critic Fine (TCCF) paradigm that allocates large block sizes for exploratory thinking and smaller block sizes for precise refinement. To stabilize training under varying block configurations, we adopt Progressive Block Size Extension, which mitigates quality degradation when scaling up block sizes. Extensive evaluations on six benchmarks show that our TDAR-8B model with BACD and TCCF achieves a 2.38 speedup and +3.4% average accuracy over the strong TraDo-8B baseline, unlocking the potential of block diffusion in complex reasoning.
OPE: Overcoming Information Saturation in Parallel Thinking via Outline-Guided Path Exploration
Qi Guo | Jianing Wang | Deyang Kong | Xiangyu Xi | Jianfei Zhang | Yi Lu | Jingang Wang | Wei Wang | Shikun Zhang | Wei Ye
Findings of the Association for Computational Linguistics: EMNLP 2026
Qi Guo | Jianing Wang | Deyang Kong | Xiangyu Xi | Jianfei Zhang | Yi Lu | Jingang Wang | Wei Wang | Shikun Zhang | Wei Ye
Findings of the Association for Computational Linguistics: EMNLP 2026
Parallel thinking has emerged as a new paradigm for large reasoning models (LRMs) in tackling complex problems. Recent methods leverage Reinforcement Learning (RL) to enhance parallel thinking, aiming to address the limitations in computational resources and effectiveness encountered with supervised fine-tuning. However, most existing studies primarily focus on optimizing the aggregation phase, with limited attention to the path exploration stage. In this paper, we theoretically analyze the optimization of parallel thinking under the Reinforcement Learning with Verifiable Rewards (RLVR) setting, and identify that the mutual information bottleneck among exploration paths fundamentally restricts overall performance. To address this, we propose Outline-Guided Path Exploration (OPE), which explicitly partitions the solution space by generating diverse reasoning outlines prior to parallel path reasoning, thereby reducing information redundancy and improving the diversity of information captured across exploration paths. OPE are implemented with an iterative RL strategy that optimizes outline planning and outline-guided reasoning independently. Extensive experiments across multiple challenging mathematical benchmarks demonstrate that OPE effectively improves reasoning performance in different aggregation strategies, enabling LRMs to more reliably discover correct solutions.
2025
SampleMix: A Sample-wise Pre-training Data Mixing Strategy by Coordinating Data Quality and Diversity
Xiangyu Xi | Deyang Kong | Jian Yang | Jiawei Yang | Zhengyu Chen | Wei Wang | Jingang Wang | Xunliang Cai | Shikun Zhang | Wei Ye
Findings of the Association for Computational Linguistics: EMNLP 2025
Xiangyu Xi | Deyang Kong | Jian Yang | Jiawei Yang | Zhengyu Chen | Wei Wang | Jingang Wang | Xunliang Cai | Shikun Zhang | Wei Ye
Findings of the Association for Computational Linguistics: EMNLP 2025
Existing pretraining data mixing methods for large language models (LLMs) typically follow a domain-wise methodology, a top-down process that first determines domain weights and then performs uniform data sampling across each domain. However, these approaches neglect significant inter-domain overlaps and commonalities, failing to control the global diversity of the constructed training dataset. Further, uniform sampling within domains ignores fine-grained sample-specific features, potentially leading to suboptimal data distribution. To address these shortcomings, we propose a novel sample-wise data mixture approach based on a bottom-up paradigm. This method performs global cross-domain sampling by systematically evaluating the quality and diversity of each sample, thereby dynamically determining the optimal domain distribution. Comprehensive experiments across multiple downstream tasks and perplexity assessments demonstrate that SampleMix surpasses existing domain-based methods. Meanwhile, SampleMix requires 1.4x to 2.1x fewer training steps to achieve the baselines’ performance, highlighting the substantial potential of SampleMix to optimize pre-training data.