A Topic Augmented Text Generation Model: Joint Learning of Semantics and Structural Features

Hongyin Tang, Miao Li, Beihong Jin


Abstract
Text generation is among the most fundamental tasks in natural language processing. In this paper, we propose a text generation model that learns semantics and structural features simultaneously. This model captures structural features by a sequential variational autoencoder component and leverages a topic modeling component based on Gaussian distribution to enhance the recognition of text semantics. To make the reconstructed text more coherent to the topics, the model further adapts the encoder of the topic modeling component for a discriminator. The results of experiments over several datasets demonstrate that our model outperforms several states of the art models in terms of text perplexity and topic coherence. Moreover, the latent representations learned by our model is superior to others in a text classification task. Finally, given the input texts, our model can generate meaningful texts which hold similar structures but under different topics.
Anthology ID:
D19-1513
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
5090–5099
Language:
URL:
https://aclanthology.org/D19-1513
DOI:
10.18653/v1/D19-1513
Bibkey:
Cite (ACL):
Hongyin Tang, Miao Li, and Beihong Jin. 2019. A Topic Augmented Text Generation Model: Joint Learning of Semantics and Structural Features. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5090–5099, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
A Topic Augmented Text Generation Model: Joint Learning of Semantics and Structural Features (Tang et al., EMNLP-IJCNLP 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-1513.pdf
Attachment:
 D19-1513.Attachment.zip