Unified Neural Topic Model via Contrastive Learning and Term Weighting

Sungwon Han, Mingi Shin, Sungkyu Park, Changwook Jung, Meeyoung Cha


Abstract
Two types of topic modeling predominate: generative methods that employ probabilistic latent models and clustering methods that identify semantically coherent groups. This paper newly presents UTopic (Unified neural Topic model via contrastive learning and term weighting) that combines the advantages of these two types. UTopic uses contrastive learning and term weighting to learn knowledge from a pretrained language model and discover influential terms from semantically coherent clusters. Experiments show that the generated topics have a high-quality topic-word distribution in terms of topic coherence, outperforming existing baselines across multiple topic coherence measures. We demonstrate how our model can be used as an add-on to existing topic models and improve their performance.
Anthology ID:
2023.eacl-main.132
Volume:
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1802–1817
Language:
URL:
https://aclanthology.org/2023.eacl-main.132
DOI:
10.18653/v1/2023.eacl-main.132
Bibkey:
Cite (ACL):
Sungwon Han, Mingi Shin, Sungkyu Park, Changwook Jung, and Meeyoung Cha. 2023. Unified Neural Topic Model via Contrastive Learning and Term Weighting. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1802–1817, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
Unified Neural Topic Model via Contrastive Learning and Term Weighting (Han et al., EACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.eacl-main.132.pdf
Video:
 https://aclanthology.org/2023.eacl-main.132.mp4