Scheduled Multi-Task Learning: From Syntax to Translation

Eliyahu Kiperwasser, Miguel Ballesteros


Abstract
Neural encoder-decoder models of machine translation have achieved impressive results, while learning linguistic knowledge of both the source and target languages in an implicit end-to-end manner. We propose a framework in which our model begins learning syntax and translation interleaved, gradually putting more focus on translation. Using this approach, we achieve considerable improvements in terms of BLEU score on relatively large parallel corpus (WMT14 English to German) and a low-resource (WIT German to English) setup.
Anthology ID:
Q18-1017
Volume:
Transactions of the Association for Computational Linguistics, Volume 6
Month:
Year:
2018
Address:
Cambridge, MA
Editors:
Lillian Lee, Mark Johnson, Kristina Toutanova, Brian Roark
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
225–240
Language:
URL:
https://aclanthology.org/Q18-1017
DOI:
10.1162/tacl_a_00017
Bibkey:
Cite (ACL):
Eliyahu Kiperwasser and Miguel Ballesteros. 2018. Scheduled Multi-Task Learning: From Syntax to Translation. Transactions of the Association for Computational Linguistics, 6:225–240.
Cite (Informal):
Scheduled Multi-Task Learning: From Syntax to Translation (Kiperwasser & Ballesteros, TACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/Q18-1017.pdf
Data
Penn Treebank