Bowen Ren
Author directory2026
Demystifying Entropy Conflict: Why Reasoning Compression Stalls in Large Reasoning Models
Hourun Zhu | Yang Gao | Wenlong Fei | Jiawei Li | Huashan Sun | Bowen Ren | Chengzhi Hu
Findings of the Association for Computational Linguistics: EMNLP 2026
Hourun Zhu | Yang Gao | Wenlong Fei | Jiawei Li | Huashan Sun | Bowen Ren | Chengzhi Hu
Findings of the Association for Computational Linguistics: EMNLP 2026
Large reasoning models achieve strong problem-solving performance, but often generate excessively long chain-of-thought outputs, which substantially increase inference cost and limit practical deployment. Existing reasoning compression methods reduce length only partially and often suffer from unstable optimization or accuracy degradation. We trace this limitation to an underlying trajectory-level objective conflict: when the compression objective and the accuracy objective are optimized jointly, compression favors short-and-correct trajectories while accuracy optimization continues to preserve long-but-correct ones that remain useful on hard problems. This conflict creates opposing gradients on the same behaviors and can trap training in a local trade-off where further compression harms accuracy while accuracy recovery reintroduces verbose reasoning. At the model-decision level, we refer to the resulting failure mode as entropy conflict: the policy fails to fully collapse onto concise trajectories because accuracy-oriented updates keep restoring broader support over long-correct reasoning paths. To address this issue, we propose an entropy-guided training framework. Instead of forcing a mixed objective to follow an unstable path toward the final efficient-output set, we first contract the policy to a short-output set and then expand from this compact regime to recover accuracy and robustness. Experiments on multiple mathematical reasoning benchmarks show that our method achieves performance comparable to the original model using only about 25% of its reasoning tokens.
2024
Fundamental Capabilities of Large Language Models and their Applications in Domain Scenarios: A Survey
Jiawei Li | Yizhe Yang | Yu Bai | Xiaofeng Zhou | Yinghao Li | Huashan Sun | Yuhang Liu | Xingpeng Si | Yuhao Ye | Yixiao Wu | Yiguan Lin | Bin Xu | Bowen Ren | Chong Feng | Yang Gao | Heyan Huang
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Jiawei Li | Yizhe Yang | Yu Bai | Xiaofeng Zhou | Yinghao Li | Huashan Sun | Yuhang Liu | Xingpeng Si | Yuhao Ye | Yixiao Wu | Yiguan Lin | Bin Xu | Bowen Ren | Chong Feng | Yang Gao | Heyan Huang
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Large Language Models (LLMs) demonstrate significant value in domain-specific applications, benefiting from their fundamental capabilities. Nevertheless, it is still unclear which fundamental capabilities contribute to success in specific domains. Moreover, the existing benchmark-based evaluation cannot effectively reflect the performance of real-world applications. In this survey, we review recent advances of LLMs in domain applications, aiming to summarize the fundamental capabilities and their collaboration. Furthermore, we establish connections between fundamental capabilities and specific domains, evaluating the varying importance of different capabilities. Based on our findings, we propose a reliable strategy for domains to choose more robust backbone LLMs for real-world applications.