Improving historical spelling normalization with bi-directional LSTMs and multi-task learning

Marcel Bollmann, Anders Søgaard


Abstract
Natural-language processing of historical documents is complicated by the abundance of variant spellings and lack of annotated data. A common approach is to normalize the spelling of historical words to modern forms. We explore the suitability of a deep neural network architecture for this task, particularly a deep bi-LSTM network applied on a character level. Our model compares well to previously established normalization algorithms when evaluated on a diverse set of texts from Early New High German. We show that multi-task learning with additional normalization data can improve our model’s performance further.
Anthology ID:
C16-1013
Volume:
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
Month:
December
Year:
2016
Address:
Osaka, Japan
Editors:
Yuji Matsumoto, Rashmi Prasad
Venue:
COLING
SIG:
Publisher:
The COLING 2016 Organizing Committee
Note:
Pages:
131–139
Language:
URL:
https://aclanthology.org/C16-1013
DOI:
Bibkey:
Cite (ACL):
Marcel Bollmann and Anders Søgaard. 2016. Improving historical spelling normalization with bi-directional LSTMs and multi-task learning. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 131–139, Osaka, Japan. The COLING 2016 Organizing Committee.
Cite (Informal):
Improving historical spelling normalization with bi-directional LSTMs and multi-task learning (Bollmann & Søgaard, COLING 2016)
Copy Citation:
PDF:
https://aclanthology.org/C16-1013.pdf
Presentation:
 C16-1013.Presentation.pdf