Unsupervised Extraction of Partial Translations for Neural Machine Translation

Benjamin Marie, Atsushi Fujita


Abstract
In neural machine translation (NMT), monolingual data are usually exploited through a so-called back-translation: sentences in the target language are translated into the source language to synthesize new parallel data. While this method provides more training data to better model the target language, on the source side, it only exploits translations that the NMT system is already able to generate using a model trained on existing parallel data. In this work, we assume that new translation knowledge can be extracted from monolingual data, without relying at all on existing parallel data. We propose a new algorithm for extracting from monolingual data what we call partial translations: pairs of source and target sentences that contain sequences of tokens that are translations of each other. Our algorithm is fully unsupervised and takes only source and target monolingual data as input. Our empirical evaluation points out that our partial translations can be used in combination with back-translation to further improve NMT models. Furthermore, while partial translations are particularly useful for low-resource language pairs, they can also be successfully exploited in resource-rich scenarios to improve translation quality.
Anthology ID:
N19-1384
Volume:
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Jill Burstein, Christy Doran, Thamar Solorio
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3834–3844
Language:
URL:
https://aclanthology.org/N19-1384
DOI:
10.18653/v1/N19-1384
Bibkey:
Cite (ACL):
Benjamin Marie and Atsushi Fujita. 2019. Unsupervised Extraction of Partial Translations for Neural Machine Translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3834–3844, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Unsupervised Extraction of Partial Translations for Neural Machine Translation (Marie & Fujita, NAACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/N19-1384.pdf