Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets

Zhen Yang, Wei Chen, Feng Wang, Bo Xu


Abstract
This paper proposes an approach for applying GANs to NMT. We build a conditional sequence generative adversarial net which comprises of two adversarial sub models, a generator and a discriminator. The generator aims to generate sentences which are hard to be discriminated from human-translated sentences ( i.e., the golden target sentences); And the discriminator makes efforts to discriminate the machine-generated sentences from human-translated ones. The two sub models play a mini-max game and achieve the win-win situation when they reach a Nash Equilibrium. Additionally, the static sentence-level BLEU is utilized as the reinforced objective for the generator, which biases the generation towards high BLEU points. During training, both the dynamic discriminator and the static BLEU objective are employed to evaluate the generated sentences and feedback the evaluations to guide the learning of the generator. Experimental results show that the proposed model consistently outperforms the traditional RNNSearch and the newly emerged state-of-the-art Transformer on English-German and Chinese-English translation tasks.
Anthology ID:
N18-1122
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)
Month:
June
Year:
2018
Address:
New Orleans, Louisiana
Editors:
Marilyn Walker, Heng Ji, Amanda Stent
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1346–1355
Language:
URL:
https://aclanthology.org/N18-1122
DOI:
10.18653/v1/N18-1122
Bibkey:
Cite (ACL):
Zhen Yang, Wei Chen, Feng Wang, and Bo Xu. 2018. Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1346–1355, New Orleans, Louisiana. Association for Computational Linguistics.
Cite (Informal):
Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets (Yang et al., NAACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/N18-1122.pdf
Note:
 N18-1122.Notes.pdf
Code
 ZhenYangIACAS/NMT_GAN +  additional community code