CRASpell: A Contextual Typo Robust Approach to Improve Chinese Spelling Correction

Shulin Liu, Shengkang Song, Tianchi Yue, Tao Yang, Huihui Cai, TingHao Yu, Shengli Sun


Abstract
Recently, Bert-based models have dominated the research of Chinese spelling correction (CSC). These methods have two limitations: (1) they have poor performance on multi-typo texts. In such texts, the context of each typo contains at least one misspelled character, which brings noise information. Such noisy context leads to the declining performance on multi-typo texts. (2) they tend to overcorrect valid expressions to more frequent expressions due to the masked token recovering task of Bert. We attempt to address these limitations in this paper. To make our model robust to contextual noise brought by typos, our approach first constructs a noisy context for each training sample. Then the correction model is forced to yield similar outputs based on the noisy and original contexts. Moreover, to address the overcorrection problem, copy mechanism is incorporated to encourage our model to prefer to choose the input character when the miscorrected and input character are both valid according to the given context. Experiments are conducted on widely used benchmarks. Our model achieves superior performance against state-of-the-art methods by a remarkable gain.
Anthology ID:
2022.findings-acl.237
Volume:
Findings of the Association for Computational Linguistics: ACL 2022
Month:
May
Year:
2022
Address:
Dublin, Ireland
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3008–3018
Language:
URL:
https://aclanthology.org/2022.findings-acl.237
DOI:
10.18653/v1/2022.findings-acl.237
Bibkey:
Cite (ACL):
Shulin Liu, Shengkang Song, Tianchi Yue, Tao Yang, Huihui Cai, TingHao Yu, and Shengli Sun. 2022. CRASpell: A Contextual Typo Robust Approach to Improve Chinese Spelling Correction. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3008–3018, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
CRASpell: A Contextual Typo Robust Approach to Improve Chinese Spelling Correction (Liu et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-acl.237.pdf
Software:
 2022.findings-acl.237.software.zip
Code
 liushulinle/craspell