Hunter NMT System for WMT18 Biomedical Translation Task: Transfer Learning in Neural Machine Translation

Abdul Khan, Subhadarshi Panda, Jia Xu, Lampros Flokas


Abstract
This paper describes the submission of Hunter Neural Machine Translation (NMT) to the WMT’18 Biomedical translation task from English to French. The discrepancy between training and test data distribution brings a challenge to translate text in new domains. Beyond the previous work of combining in-domain with out-of-domain models, we found accuracy and efficiency gain in combining different in-domain models. We conduct extensive experiments on NMT with transfer learning. We train on different in-domain Biomedical datasets one after another. That means parameters of the previous training serve as the initialization of the next one. Together with a pre-trained out-of-domain News model, we enhanced translation quality with 3.73 BLEU points over the baseline. Furthermore, we applied ensemble learning on training models of intermediate epochs and achieved an improvement of 4.02 BLEU points over the baseline. Overall, our system is 11.29 BLEU points above the best system of last year on the EDP 2017 test set.
Anthology ID:
W18-6447
Volume:
Proceedings of the Third Conference on Machine Translation: Shared Task Papers
Month:
October
Year:
2018
Address:
Belgium, Brussels
Editors:
Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Matt Post, Lucia Specia, Marco Turchi, Karin Verspoor
Venue:
WMT
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
655–661
Language:
URL:
https://aclanthology.org/W18-6447
DOI:
10.18653/v1/W18-6447
Bibkey:
Cite (ACL):
Abdul Khan, Subhadarshi Panda, Jia Xu, and Lampros Flokas. 2018. Hunter NMT System for WMT18 Biomedical Translation Task: Transfer Learning in Neural Machine Translation. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 655–661, Belgium, Brussels. Association for Computational Linguistics.
Cite (Informal):
Hunter NMT System for WMT18 Biomedical Translation Task: Transfer Learning in Neural Machine Translation (Khan et al., WMT 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-6447.pdf