Tree-Structured Topic Modeling with Nonparametric Neural Variational Inference

Ziye Chen, Cheng Ding, Zusheng Zhang, Yanghui Rao, Haoran Xie


Abstract
Topic modeling has been widely used for discovering the latent semantic structure of documents, but most existing methods learn topics with a flat structure. Although probabilistic models can generate topic hierarchies by introducing nonparametric priors like Chinese restaurant process, such methods have data scalability issues. In this study, we develop a tree-structured topic model by leveraging nonparametric neural variational inference. Particularly, the latent components of the stick-breaking process are first learned for each document, then the affiliations of latent components are modeled by the dependency matrices between network layers. Utilizing this network structure, we can efficiently extract a tree-structured topic hierarchy with reasonable structure, low redundancy, and adaptable widths. Experiments on real-world datasets validate the effectiveness of our method.
Anthology ID:
2021.acl-long.182
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
August
Year:
2021
Address:
Online
Editors:
Chengqing Zong, Fei Xia, Wenjie Li, Roberto Navigli
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2343–2353
Language:
URL:
https://aclanthology.org/2021.acl-long.182
DOI:
10.18653/v1/2021.acl-long.182
Bibkey:
Cite (ACL):
Ziye Chen, Cheng Ding, Zusheng Zhang, Yanghui Rao, and Haoran Xie. 2021. Tree-Structured Topic Modeling with Nonparametric Neural Variational Inference. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2343–2353, Online. Association for Computational Linguistics.
Cite (Informal):
Tree-Structured Topic Modeling with Nonparametric Neural Variational Inference (Chen et al., ACL-IJCNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-long.182.pdf
Video:
 https://aclanthology.org/2021.acl-long.182.mp4
Code
 hostnlp/ntsntm