Long Warm-up and Self-Training: Training Strategies of NICT-2 NMT System at WAT-2019

Kenji Imamura, Eiichiro Sumita


Abstract
This paper describes the NICT-2 neural machine translation system at the 6th Workshop on Asian Translation. This system employs the standard Transformer model but features the following two characteristics. One is the long warm-up strategy, which performs a longer warm-up of the learning rate at the start of the training than conventional approaches. Another is that the system introduces self-training approaches based on multiple back-translations generated by sampling. We participated in three tasks—ASPEC.en-ja, ASPEC.ja-en, and TDDC.ja-en—using this system.
Anthology ID:
D19-5217
Volume:
Proceedings of the 6th Workshop on Asian Translation
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Toshiaki Nakazawa, Chenchen Ding, Raj Dabre, Anoop Kunchukuttan, Nobushige Doi, Yusuke Oda, Ondřej Bojar, Shantipriya Parida, Isao Goto, Hidaya Mino
Venue:
WAT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
141–146
Language:
URL:
https://aclanthology.org/D19-5217
DOI:
10.18653/v1/D19-5217
Bibkey:
Cite (ACL):
Kenji Imamura and Eiichiro Sumita. 2019. Long Warm-up and Self-Training: Training Strategies of NICT-2 NMT System at WAT-2019. In Proceedings of the 6th Workshop on Asian Translation, pages 141–146, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Long Warm-up and Self-Training: Training Strategies of NICT-2 NMT System at WAT-2019 (Imamura & Sumita, WAT 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-5217.pdf