Nozomu Miyamoto
2023
Dynamic Structured Neural Topic Model with Self-Attention Mechanism
Nozomu Miyamoto
|
Masaru Isonuma
|
Sho Takase
|
Junichiro Mori
|
Ichiro Sakata
Findings of the Association for Computational Linguistics: ACL 2023
This study presents a dynamic structured neural topic model, which can handle the time-series development of topics while capturing their dependencies. Our model captures the topic branching and merging processes by modeling topic dependencies based on a self-attention mechanism. Additionally, we introduce citation regularization, which induces attention weights to represent citation relations by modeling text and citations jointly. Our model outperforms a prior dynamic embedded topic model regarding perplexity and coherence, while maintaining sufficient diversity across topics. Furthermore, we confirm that our model can potentially predict emerging topics from academic literature.
Search