Continuous Space Reordering Models for Phrase-based MT

Nadir Durrani, Fahim Dalvi


Abstract
Bilingual sequence models improve phrase-based translation and reordering by overcoming phrasal independence assumption and handling long range reordering. However, due to data sparsity, these models often fall back to very small context sizes. This problem has been previously addressed by learning sequences over generalized representations such as POS tags or word clusters. In this paper, we explore an alternative based on neural network models. More concretely we train neuralized versions of lexicalized reordering [1] and the operation sequence models [2] using feed-forward neural network. Our results show improvements of up to 0.6 and 0.5 BLEU points on top of the baseline German!English and English!German systems. We also observed improvements compared to the systems that used POS tags and word clusters to train these models. Because we modify the bilingual corpus to integrate reordering operations, this allows us to also train a sequence-to-sequence neural MT model having explicit reordering triggers. Our motivation was to directly enable reordering information in the encoder-decoder framework, which otherwise relies solely on the attention model to handle long range reordering. We tried both coarser and fine-grained reordering operations. However, these experiments did not yield any improvements over the baseline Neural MT systems.
Anthology ID:
2017.iwslt-1.18
Volume:
Proceedings of the 14th International Conference on Spoken Language Translation
Month:
December 14-15
Year:
2017
Address:
Tokyo, Japan
Venue:
IWSLT
SIG:
SIGSLT
Publisher:
International Workshop on Spoken Language Translation
Note:
Pages:
129–136
Language:
URL:
https://aclanthology.org/2017.iwslt-1.18
DOI:
Bibkey:
Cite (ACL):
Nadir Durrani and Fahim Dalvi. 2017. Continuous Space Reordering Models for Phrase-based MT. In Proceedings of the 14th International Conference on Spoken Language Translation, pages 129–136, Tokyo, Japan. International Workshop on Spoken Language Translation.
Cite (Informal):
Continuous Space Reordering Models for Phrase-based MT (Durrani & Dalvi, IWSLT 2017)
Copy Citation:
PDF:
https://aclanthology.org/2017.iwslt-1.18.pdf