Duo Xu
Author directory2026
APEX : Adaptively Balancing Off-Policy Guidance and On-Policy Exploration for LLM Reasoning
Yuchen Wang | Zikang Zhang | Duo Xu | Kunpeng Xu | Qihui Zhu | Wei Huang | Siying Wu | Caihong Cheng | Xin He | Yang Yang | Guanjun Jiang | Xiaoyan Sun
Findings of the Association for Computational Linguistics: EMNLP 2026
Yuchen Wang | Zikang Zhang | Duo Xu | Kunpeng Xu | Qihui Zhu | Wei Huang | Siying Wu | Caihong Cheng | Xin He | Yang Yang | Guanjun Jiang | Xiaoyan Sun
Findings of the Association for Computational Linguistics: EMNLP 2026
In post-training large language models (LLMs), supervised fine-tuning (SFT) provides expert guidance but can overfit, while reinforcement learning (RL) enables exploration but often suffers from cold-start and sparse-reward issues. Although combining SFT with RL is intuitive, naively mixing off-policy expert data into on-policy RL can induce distribution shift and destabilize optimization. We propose APEX, a framework that harmonizes off-policy guidance and on-policy exploration via a Cognitive State Awareness mechanism. Specifically, APEX leverages a perplexity-based Student’s -test to compare the sequence-level perplexity distributions of on-policy generations against candidate offline data. This comparison diagnoses whether failures reflect sampling stochasticity or missing knowledge. Guided by this diagnosis, APEX adaptively gates GRPO optimization and modulates SFT supervision. The framework preferentially selects offline samples that are statistically consistent with the current on-policy distribution to guide RL updates, while filtering out high-shift data and verifiably incorrect samples that could destabilize training. APEX also uses a dynamically weighted SFT loss when needed. Extensive experiments show that APEX improves over SFT-then-RL by +7.10 points in average pass@1 on six math benchmarks and by +15.50 points on three general reasoning benchmarks.
2025
ChartM3: A Multi-Stage Code-Driven Pipeline for Constructing Multi-Dimensional and Multi-Step Visual Reasoning Data in Chart Comprehension
Duo Xu | Hao Cheng | Xin Lin | Zhen Xie | Hao Henry Wang
Findings of the Association for Computational Linguistics: EMNLP 2025
Duo Xu | Hao Cheng | Xin Lin | Zhen Xie | Hao Henry Wang
Findings of the Association for Computational Linguistics: EMNLP 2025
Complex chart understanding tasks demand advanced visual recognition and reasoning capabilities from multimodal large language models (MLLMs). However, current research provides limited coverage of complex chart scenarios and computation-intensive reasoning tasks prevalent in real-world applications. This study proposes an automated multi-stage code-driven pipeline for systematically generating visual reasoning datasets to address these limitations. The pipeline integrates retrieval-augmented generation (RAG) to retrieve professional chart templates and employs chain-of-thought (CoT) strategies to generate reasoning codes that simulate real data distributions, thereby driving chart rendering and question-related statistical computations. Through model-based evaluation, the pipeline enhances chart diversity and data quality. Using this framework, we construct ChartM3, a multi-dimensional and multi-step dataset containing 38K charts and 142K Q&A pairs for training, along with 2,871 high-quality evaluation samples for enabling practical performance assessment. Supervised fine-tuning (SFT) and reinforcement learning (RL) experiments demonstrate that our dataset significantly improves reasoning capabilities and cross-domain generalization performance, enabling smaller models to achieve performance comparable to larger-scale models in complex chart comprehension.