Weighted self Distillation for Chinese word segmentation

Rian He, Shubin Cai, Zhong Ming, Jialei Zhang


Abstract
Recent researches show that multi-criteria resources and n-gram features are beneficial to Chinese Word Segmentation (CWS). However, these methods rely heavily on such additional information mentioned above and focus less on the model itself. We thus propose a novel neural framework, named Weighted self Distillation for Chinese word segmentation (WeiDC). The framework, which only requires unigram features, adopts self-distillation technology with four hand-crafted weight modules and two teacher models configurations. Experiment results show that WeiDC can make use of character features to learn contextual knowledge and successfully achieve state-of-the-art or competitive performance in terms of strictly closed test settings on SIGHAN Bakeoff benchmark datasets. Moreover, further experiments and analyses also demonstrate the robustness of WeiDC. Source codes of this paper are available on Github.
Anthology ID:
2022.findings-acl.139
Volume:
Findings of the Association for Computational Linguistics: ACL 2022
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1757–1770
Language:
URL:
https://aclanthology.org/2022.findings-acl.139
DOI:
10.18653/v1/2022.findings-acl.139
Bibkey:
Cite (ACL):
Rian He, Shubin Cai, Zhong Ming, and Jialei Zhang. 2022. Weighted self Distillation for Chinese word segmentation. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1757–1770, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Weighted self Distillation for Chinese word segmentation (He et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-acl.139.pdf
Software:
 2022.findings-acl.139.software.zip
Code
 anzi20/weidc