Multi-task learning for historical text normalization: Size matters

Marcel Bollmann, Anders Søgaard, Joachim Bingel


Abstract
Historical text normalization suffers from small datasets that exhibit high variance, and previous work has shown that multi-task learning can be used to leverage data from related problems in order to obtain more robust models. Previous work has been limited to datasets from a specific language and a specific historical period, and it is not clear whether results generalize. It therefore remains an open problem, when historical text normalization benefits from multi-task learning. We explore the benefits of multi-task learning across 10 different datasets, representing different languages and periods. Our main finding—contrary to what has been observed for other NLP tasks—is that multi-task learning mainly works when target task data is very scarce.
Anthology ID:
W18-3403
Volume:
Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP
Month:
July
Year:
2018
Address:
Melbourne
Editors:
Reza Haffari, Colin Cherry, George Foster, Shahram Khadivi, Bahar Salehi
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
19–24
Language:
URL:
https://aclanthology.org/W18-3403
DOI:
10.18653/v1/W18-3403
Bibkey:
Cite (ACL):
Marcel Bollmann, Anders Søgaard, and Joachim Bingel. 2018. Multi-task learning for historical text normalization: Size matters. In Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP, pages 19–24, Melbourne. Association for Computational Linguistics.
Cite (Informal):
Multi-task learning for historical text normalization: Size matters (Bollmann et al., ACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-3403.pdf
Poster:
 W18-3403.Poster.pdf
Data
FCE