FinBART: A Pre-trained Seq2seq Language Model for Chinese Financial Tasks

Dong Hongyuan, Che Wanxiang, He Xiaoyu, Zheng Guidong, Wen Junjie


Abstract
“Pretrained language models are making a more profound impact on our lives than ever before. They exhibit promising performance on a variety of general domain Natural Language Process-ing (NLP) tasks. However, few work focuses on Chinese financial NLP tasks, which comprisea significant portion of social communication. To this end, we propose FinBART, a pretrainedseq2seq language model for Chinese financial communication tasks. Experiments show thatFinBART outperforms baseline models on a series of downstream tasks including text classifica-tion, sequence labeling and text generation. We further pretrain the model on customer servicecorpora, and results show that our model outperforms baseline models and achieves promisingperformance on various real world customer service text mining tasks.”
Anthology ID:
2023.ccl-1.77
Volume:
Proceedings of the 22nd Chinese National Conference on Computational Linguistics
Month:
August
Year:
2023
Address:
Harbin, China
Editors:
Maosong Sun, Bing Qin, Xipeng Qiu, Jing Jiang, Xianpei Han
Venue:
CCL
SIG:
Publisher:
Chinese Information Processing Society of China
Note:
Pages:
906–917
Language:
English
URL:
https://aclanthology.org/2023.ccl-1.77
DOI:
Bibkey:
Cite (ACL):
Dong Hongyuan, Che Wanxiang, He Xiaoyu, Zheng Guidong, and Wen Junjie. 2023. FinBART: A Pre-trained Seq2seq Language Model for Chinese Financial Tasks. In Proceedings of the 22nd Chinese National Conference on Computational Linguistics, pages 906–917, Harbin, China. Chinese Information Processing Society of China.
Cite (Informal):
FinBART: A Pre-trained Seq2seq Language Model for Chinese Financial Tasks (Hongyuan et al., CCL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.ccl-1.77.pdf