An Evaluation of Neural Machine Translation Models on Historical Spelling Normalization

Gongbo Tang, Fabienne Cap, Eva Pettersson, Joakim Nivre


Abstract
In this paper, we apply different NMT models to the problem of historical spelling normalization for five languages: English, German, Hungarian, Icelandic, and Swedish. The NMT models are at different levels, have different attention mechanisms, and different neural network architectures. Our results show that NMT models are much better than SMT models in terms of character error rate. The vanilla RNNs are competitive to GRUs/LSTMs in historical spelling normalization. Transformer models perform better only when provided with more training data. We also find that subword-level models with a small subword vocabulary are better than character-level models. In addition, we propose a hybrid method which further improves the performance of historical spelling normalization.
Anthology ID:
C18-1112
Volume:
Proceedings of the 27th International Conference on Computational Linguistics
Month:
August
Year:
2018
Address:
Santa Fe, New Mexico, USA
Editors:
Emily M. Bender, Leon Derczynski, Pierre Isabelle
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1320–1331
Language:
URL:
https://aclanthology.org/C18-1112
DOI:
Bibkey:
Cite (ACL):
Gongbo Tang, Fabienne Cap, Eva Pettersson, and Joakim Nivre. 2018. An Evaluation of Neural Machine Translation Models on Historical Spelling Normalization. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1320–1331, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Cite (Informal):
An Evaluation of Neural Machine Translation Models on Historical Spelling Normalization (Tang et al., COLING 2018)
Copy Citation:
PDF:
https://aclanthology.org/C18-1112.pdf
Code
 tanggongbo/normalization-NMT