On the Properties of Neural Machine Translation: Encoder–Decoder Approaches

Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, Yoshua Bengio


Abstract
Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates a correct translation from this representation. In this paper, we focus on analyzing the properties of the neural machine translation using two models; RNN Encoder–Decoder and a newly proposed gated recursive convolutional neural network. We show that the neural machine translation performs relatively well on short sentences without unknown words, but its performance degrades rapidly as the length of the sentence and the number of unknown words increase. Furthermore, we find that the proposed gated recursive convolutional network learns a grammatical structure of a sentence automatically.
Anthology ID:
W14-4012
Volume:
Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation
Month:
October
Year:
2014
Address:
Doha, Qatar
Editors:
Dekai Wu, Marine Carpuat, Xavier Carreras, Eva Maria Vecchi
Venues:
SSST | WS
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
103–111
Language:
URL:
https://aclanthology.org/W14-4012/
DOI:
10.3115/v1/W14-4012
Bibkey:
Cite (ACL):
Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the Properties of Neural Machine Translation: Encoder–Decoder Approaches. In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, pages 103–111, Doha, Qatar. Association for Computational Linguistics.
Cite (Informal):
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches (Cho et al., SSST 2014)
Copy Citation:
PDF:
https://aclanthology.org/W14-4012.pdf