ARAML: A Stable Adversarial Training Framework for Text Generation

Pei Ke, Fei Huang, Minlie Huang, Xiaoyan Zhu


Abstract
Most of the existing generative adversarial networks (GAN) for text generation suffer from the instability of reinforcement learning training algorithms such as policy gradient, leading to unstable performance. To tackle this problem, we propose a novel framework called Adversarial Reward Augmented Maximum Likelihood (ARAML). During adversarial training, the discriminator assigns rewards to samples which are acquired from a stationary distribution near the data rather than the generator’s distribution. The generator is optimized with maximum likelihood estimation augmented by the discriminator’s rewards instead of policy gradient. Experiments show that our model can outperform state-of-the-art text GANs with a more stable training process.
Anthology ID:
D19-1436
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
4271–4281
Language:
URL:
https://aclanthology.org/D19-1436
DOI:
10.18653/v1/D19-1436
Bibkey:
Cite (ACL):
Pei Ke, Fei Huang, Minlie Huang, and Xiaoyan Zhu. 2019. ARAML: A Stable Adversarial Training Framework for Text Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4271–4281, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
ARAML: A Stable Adversarial Training Framework for Text Generation (Ke et al., EMNLP 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-1436.pdf
Code
 kepei1106/ARAML