Multi-representation ensembles and delayed SGD updates improve syntax-based NMT

Danielle Saunders, Felix Stahlberg, Adrià de Gispert, Bill Byrne


Abstract
We explore strategies for incorporating target syntax into Neural Machine Translation. We specifically focus on syntax in ensembles containing multiple sentence representations. We formulate beam search over such ensembles using WFSTs, and describe a delayed SGD update training procedure that is especially effective for long representations like linearized syntax. Our approach gives state-of-the-art performance on a difficult Japanese-English task.
Anthology ID:
P18-2051
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Iryna Gurevych, Yusuke Miyao
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
319–325
Language:
URL:
https://aclanthology.org/P18-2051
DOI:
10.18653/v1/P18-2051
Bibkey:
Cite (ACL):
Danielle Saunders, Felix Stahlberg, Adrià de Gispert, and Bill Byrne. 2018. Multi-representation ensembles and delayed SGD updates improve syntax-based NMT. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 319–325, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Multi-representation ensembles and delayed SGD updates improve syntax-based NMT (Saunders et al., ACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/P18-2051.pdf
Poster:
 P18-2051.Poster.pdf