Zijie Zeng
Author directory2026
GRAIL: Gradient-Reward Aligned Initialization for LoRA-based GRPO
Liwen Shi | Zijie Zeng | Bowen Wang | Yunji | Zeyu Chen | Xinyu Shao | Xinzheng Sun | Xuyang Xiong | Haoyang Wu | Sheng Zhang
Findings of the Association for Computational Linguistics: EMNLP 2026
Liwen Shi | Zijie Zeng | Bowen Wang | Yunji | Zeyu Chen | Xinyu Shao | Xinzheng Sun | Xuyang Xiong | Haoyang Wu | Sheng Zhang
Findings of the Association for Computational Linguistics: EMNLP 2026
Group Relative Policy Optimization (GRPO) improves LLM reasoning, but LoRA-based GRPO can underperform when the initialized adapter subspace is poorly aligned with reward-induced gradients. We propose GRAIL, Gradient-Reward Aligned Initialization for LoRA-based GRPO, which estimates reward-aligned gradients from grouped rollouts before policy optimization and uses them to construct LoRA factors. The full GRAIL configuration includes a reference-solution fallback only for all-zero-correctness groups during calibration; after initialization, all models use the standard GRPO objective without extra trainable parameters or loss changes. We also study Degenerate-Group Reflection Resampling (DGRR) as an optional extension that resamples all-zero-correctness groups after self-reflection without exposing reference solutions. Across Qwen2.5, Phi-3.5, and Gemma-2 backbones on GSM8K and MATH500, GRAIL consistently improves LoRA-based GRPO, with particularly strong gains on smaller Qwen and Gemma models. Analyses attribute these gains to stronger GRPO-gradient capture, smaller principal angles, and better handling of correctness-degenerate groups.
Self-Corrected Speech Reasoning: Diagnosing and Improving Condition Updating via PACER
Zijie Zeng | Liwen Shi | Yunji | Bowen Wang | Nanjie Li | Xiaoyong Guo | Zeyu Chen | Xinzheng Sun | Xuyang Xiong | Haoyang Wu | Sheng Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zijie Zeng | Liwen Shi | Yunji | Bowen Wang | Nanjie Li | Xiaoyong Guo | Zeyu Chen | Xinzheng Sun | Xuyang Xiong | Haoyang Wu | Sheng Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Speech large language models (Speech LLMs) are increasingly expected to reason in everyday spoken interactions, where users often revise task conditions through self-correction. Unlike ordinary disfluencies, self-corrections alter the premise structure of a problem: obsolete conditions must be discarded, while final corrected conditions must guide reasoning. We formulate this setting as self-corrected speech reasoning and study it as a condition-update problem. We construct SCUR-Bench (Spoken Condition-Update Reasoning Benchmark), a controlled Chinese benchmark of 13,295 instances organized by reasoning difficulty and self-correction difficulty, and introduce a three-path diagnostic protocol to analyze speech-side recovery, condition use, and reasoning. Our analysis reveals a recurrent failure pattern in current Speech LLMs: corrected conditions may be perceived, transcribed, or even mentioned, yet still fail to serve as active premises in reasoning. We term this final-condition adoption failure. A controlled English parallel evaluation exhibits the same recovery–adoption gap, providing preliminary evidence that this failure is not unique to Chinese. To mitigate it, we propose PACER, a perception-aware condition-enhanced reinforcement framework based on GRPO. On Qwen2.5-Omni-7B, PACER reduces end-to-end error and condition-related errors by 39.8% and 67.5%, respectively, relative to the untrained baseline.
2024
Overview of the 2024 ALTA Shared Task: Detect Automatic AI-Generated Sentences for Human-AI Hybrid Articles
Diego Mollá | Qiongkai Xu | Zijie Zeng | Zhuang Li
Proceedings of the 22nd Annual Workshop of the Australasian Language Technology Association
Diego Mollá | Qiongkai Xu | Zijie Zeng | Zhuang Li
Proceedings of the 22nd Annual Workshop of the Australasian Language Technology Association
The ALTA shared tasks have been running annually since 2010. In 2024, the purpose of the task is to detect machine-generated text in a hybrid setting where the text may contain portions of human text and portions machine-generated. In this paper, we present the task, the evaluation criteria, and the results of the systems participating in the shared task.
2022
Do Deep Neural Nets Display Human-like Attention in Short Answer Scoring?
Zijie Zeng | Xinyu Li | Dragan Gasevic | Guanliang Chen
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Zijie Zeng | Xinyu Li | Dragan Gasevic | Guanliang Chen
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Deep Learning (DL) techniques have been increasingly adopted for Automatic Text Scoring in education. However, these techniques often suffer from their inabilities to explain and justify how a prediction is made, which, unavoidably, decreases their trustworthiness and hinders educators from embracing them in practice. This study aimed to investigate whether (and to what extent) DL-based graders align with human graders regarding the important words they identify when marking short answer questions. To this end, we first conducted a user study to ask human graders to manually annotate important words in assessing answer quality and then measured the overlap between these human-annotated words and those identified by DL-based graders (i.e., those receiving large attention weights). Furthermore, we ran a randomized controlled experiment to explore the impact of highlighting important words detected by DL-based graders on human grading. The results showed that: (i) DL-based graders, to a certain degree, displayed alignment with human graders no matter whether DL-based graders and human graders agreed on the quality of an answer; and (ii) it is possible to facilitate human grading by highlighting those DL-detected important words, though further investigations are necessary to understand how human graders exploit such highlighted words.