FinMoE: A MoE-based Large Chinese Financial Language Model

Xuanyu Zhang, Qing Yang


Abstract
Large-scale language models have demonstrated remarkable success, achieving strong performance across a variety of general tasks. However, when applied to domain-specific fields, such as finance, these models face challenges due to the need for both specialized knowledge and robust general capabilities. In this paper, we introduce FinMoE, a MOE-based large-scale Chinese financial language model that bridges the gap between general language models and domain-specific requirements. FinMoE employs a dense MoE architecture, where all expert networks are simultaneously activated and dynamically combined to effectively integrate general linguistic understanding with domain-specific financial expertise. Experimental results demonstrate that FinMoE achieves state-of-the-art performance on both general-purpose and financial benchmarks at a comparable scale, validating its ability to balance domain specialization with general knowledge and reasoning.
Anthology ID:
2025.finnlp-1.4
Volume:
Proceedings of the Joint Workshop of the 9th Financial Technology and Natural Language Processing (FinNLP), the 6th Financial Narrative Processing (FNP), and the 1st Workshop on Large Language Models for Finance and Legal (LLMFinLegal)
Month:
January
Year:
2025
Address:
Abu Dhabi, UAE
Editors:
Chung-Chi Chen, Antonio Moreno-Sandoval, Jimin Huang, Qianqian Xie, Sophia Ananiadou, Hsin-Hsi Chen
Venues:
FinNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
48–53
Language:
URL:
https://aclanthology.org/2025.finnlp-1.4/
DOI:
Bibkey:
Cite (ACL):
Xuanyu Zhang and Qing Yang. 2025. FinMoE: A MoE-based Large Chinese Financial Language Model. In Proceedings of the Joint Workshop of the 9th Financial Technology and Natural Language Processing (FinNLP), the 6th Financial Narrative Processing (FNP), and the 1st Workshop on Large Language Models for Finance and Legal (LLMFinLegal), pages 48–53, Abu Dhabi, UAE. Association for Computational Linguistics.
Cite (Informal):
FinMoE: A MoE-based Large Chinese Financial Language Model (Zhang & Yang, FinNLP 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.finnlp-1.4.pdf