Synchronous Bidirectional Neural Machine Translation

Long Zhou, Jiajun Zhang, Chengqing Zong


Abstract
Existing approaches to neural machine translation (NMT) generate the target language sequence token-by-token from left to right. However, this kind of unidirectional decoding framework cannot make full use of the target-side future contexts which can be produced in a right-to-left decoding direction, and thus suffers from the issue of unbalanced outputs. In this paper, we introduce a synchronous bidirectional–neural machine translation (SB-NMT) that predicts its outputs using left-to-right and right-to-left decoding simultaneously and interactively, in order to leverage both of the history and future information at the same time. Specifically, we first propose a new algorithm that enables synchronous bidirectional decoding in a single model. Then, we present an interactive decoding model in which left-to-right (right-to-left) generation does not only depend on its previously generated outputs, but also relies on future contexts predicted by right-to-left (left-to-right) decoding. We extensively evaluate the proposed SB-NMT model on large-scale NIST Chinese–English, WMT14 English–German, and WMT18 Russian–English translation tasks. Experimental results demonstrate that our model achieves significant improvements over the strong Transformer model by 3.92, 1.49, and 1.04 BLEU points, respectively, and obtains the state-of-the-art performance on Chinese–English and English–German translation tasks.
Anthology ID:
Q19-1006
Volume:
Transactions of the Association for Computational Linguistics, Volume 7
Month:
Year:
2019
Address:
Cambridge, MA
Editors:
Lillian Lee, Mark Johnson, Brian Roark, Ani Nenkova
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
91–105
Language:
URL:
https://aclanthology.org/Q19-1006
DOI:
10.1162/tacl_a_00256
Bibkey:
Cite (ACL):
Long Zhou, Jiajun Zhang, and Chengqing Zong. 2019. Synchronous Bidirectional Neural Machine Translation. Transactions of the Association for Computational Linguistics, 7:91–105.
Cite (Informal):
Synchronous Bidirectional Neural Machine Translation (Zhou et al., TACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/Q19-1006.pdf
Video:
 https://aclanthology.org/Q19-1006.mp4
Code
 wszlong/sb-nmt +  additional community code
Data
WMT 2014