Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio


Abstract
In this paper, we propose a novel neural network model called RNN Encoder–Decoder that consists of two recurrent neural networks (RNN). One RNN encodes a sequence of symbols into a fixed length vector representation, and the other decodes the representation into another sequence of symbols. The encoder and decoder of the proposed model are jointly trained to maximize the conditional probability of a target sequence given a source sequence. The performance of a statistical machine translation system is empirically found to improve by using the conditional probabilities of phrase pairs computed by the RNN Encoder–Decoder as an additional feature in the existing log-linear model. Qualitatively, we show that the proposed model learns a semantically and syntactically meaningful representation of linguistic phrases.
Anthology ID:
D14-1179
Volume:
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Month:
October
Year:
2014
Address:
Doha, Qatar
Editors:
Alessandro Moschitti, Bo Pang, Walter Daelemans
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1724–1734
Language:
URL:
https://aclanthology.org/D14-1179/
DOI:
10.3115/v1/D14-1179
Bibkey:
Cite (ACL):
Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724–1734, Doha, Qatar. Association for Computational Linguistics.
Cite (Informal):
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation (Cho et al., EMNLP 2014)
Copy Citation:
PDF:
https://aclanthology.org/D14-1179.pdf
Attachment:
 D14-1179.Attachment.pdf