Benefits of Data Augmentation for NMT-based Text Normalization of User-Generated Content

Claudia Matos Veliz, Orphee De Clercq, Veronique Hoste


Abstract
One of the most persistent characteristics of written user-generated content (UGC) is the use of non-standard words. This characteristic contributes to an increased difficulty to automatically process and analyze UGC. Text normalization is the task of transforming lexical variants to their canonical forms and is often used as a pre-processing step for conventional NLP tasks in order to overcome the performance drop that NLP systems experience when applied to UGC. In this work, we follow a Neural Machine Translation approach to text normalization. To train such an encoder-decoder model, large parallel training corpora of sentence pairs are required. However, obtaining large data sets with UGC and their normalized version is not trivial, especially for languages other than English. In this paper, we explore how to overcome this data bottleneck for Dutch, a low-resource language. We start off with a small publicly available parallel Dutch data set comprising three UGC genres and compare two different approaches. The first is to manually normalize and add training data, a money and time-consuming task. The second approach is a set of data augmentation techniques which increase data size by converting existing resources into synthesized non-standard forms. Our results reveal that, while the different approaches yield similar results regarding the normalization issues in the test set, they also introduce a large amount of over-normalizations.
Anthology ID:
D19-5536
Volume:
Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Wei Xu, Alan Ritter, Tim Baldwin, Afshin Rahimi
Venue:
WNUT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
275–285
Language:
URL:
https://aclanthology.org/D19-5536
DOI:
10.18653/v1/D19-5536
Bibkey:
Cite (ACL):
Claudia Matos Veliz, Orphee De Clercq, and Veronique Hoste. 2019. Benefits of Data Augmentation for NMT-based Text Normalization of User-Generated Content. In Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019), pages 275–285, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Benefits of Data Augmentation for NMT-based Text Normalization of User-Generated Content (Matos Veliz et al., WNUT 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-5536.pdf