Multilingual Chain of Thought Compression via Cross-lingual Entropy Gap

Jiarui Wan, Yufeng Chen, Songming Zhang, Rui Qi, Xue Zhang, Jinan Xu, Kaiyu Huang


Abstract
Current large reasoning models (LRMs) prefer to generate verbose chain-of-thought (CoT) traces (a.k.a. the “overthinking” issue), especially in multilingual scenarios due to the imbalanced proficiency across different languages. However, existing studies on CoT compression focus exclusively on English and overlook language characteristics, resulting in performance degradation in multilingual scenarios. To address this issue, we develop a multilingual CoT compression method based on reinforcement learning, named Language-aware Entropy-based (LAE), which optimizes LLMs with multilingual group sampling and entropy gap-based rewards. Specifically, we extend GRPO with multilingual group sampling, where each query is jointly optimized across multiple languages. We further design a language-aware entropy-gap reward that penalizes redundant uncertainty in the thinking phase and aligns reasoning complexity across languages, enabling concise yet accurate multilingual CoT reasoning. Extensive experiments on the multilingual reasoning benchmarks MGSM, MMATH and MMMLU demonstrate that our method achieves competitive performance and generalization, compared with several strong baselines.
Anthology ID:
2026.emnlp-main.2539
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
57744–57764
Language:
URL:
https://aclanthology.org/2026.emnlp-main.2539/
DOI:
Bibkey:
Cite (ACL):
Jiarui Wan, Yufeng Chen, Songming Zhang, Rui Qi, Xue Zhang, Jinan Xu, and Kaiyu Huang. 2026. Multilingual Chain of Thought Compression via Cross-lingual Entropy Gap. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 57744–57764, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Multilingual Chain of Thought Compression via Cross-lingual Entropy Gap (Wan et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.2539.pdf