Compositional Source Word Representations for Neural Machine Translation

Duygu Ataman, Mattia Antonino Di Gangi, Marcello Federico


Abstract
The requirement for neural machine translation (NMT) models to use fixed-size input and output vocabularies plays an important role for their accuracy and generalization capability. The conventional approach to cope with this limitation is performing translation based on a vocabulary of sub-word units that are predicted using statistical word segmentation methods. However, these methods have recently shown to be prone to morphological errors, which lead to inaccurate translations. In this paper, we extend the source-language embedding layer of the NMT model with a bi-directional recurrent neural network that generates compositional representations of the source words from embeddings of character n-grams. Our model consistently outperforms conventional NMT with sub-word units on four translation directions with varying degrees of morphological complexity and data sparseness on the source side.
Anthology ID:
2018.eamt-main.3
Volume:
Proceedings of the 21st Annual Conference of the European Association for Machine Translation
Month:
May
Year:
2018
Address:
Alicante, Spain
Editors:
Juan Antonio Pérez-Ortiz, Felipe Sánchez-Martínez, Miquel Esplà-Gomis, Maja Popović, Celia Rico, André Martins, Joachim Van den Bogaert, Mikel L. Forcada
Venue:
EAMT
SIG:
Publisher:
Note:
Pages:
51–60
Language:
URL:
https://aclanthology.org/2018.eamt-main.3
DOI:
Bibkey:
Cite (ACL):
Duygu Ataman, Mattia Antonino Di Gangi, and Marcello Federico. 2018. Compositional Source Word Representations for Neural Machine Translation. In Proceedings of the 21st Annual Conference of the European Association for Machine Translation, pages 51–60, Alicante, Spain.
Cite (Informal):
Compositional Source Word Representations for Neural Machine Translation (Ataman et al., EAMT 2018)
Copy Citation:
PDF:
https://aclanthology.org/2018.eamt-main.3.pdf