Beyond Top Words: MonoTM for Topic Modeling with Interpretable Monosemantic Features

Una Joh, Bei Yu


Abstract
Topic models summarize large text corpora, but top-ranked words often provide only a limited representation of topic semantics. Sparse autoencoders (SAEs) offer a way to move beyond word-level descriptors by extracting interpretable features from dense representations, yet how feature interpretability relates to topic-inference quality remains unclear. We introduce MonoTM, an interpretable topic modeling framework that decouples these roles. Across three benchmark corpora, we show that document–topic mixture estimation and semantic interpretation favor different SAE configurations and feature subsets. MonoTM estimates mixtures from the full SAE bag-of-features representation and, with them fixed, learns topic descriptors over a separate vocabulary of corpus-grounded semantic features. This design preserves global topic structure while representing topics with semantic units more meaningful than individual words, making them more useful for downstream corpus analysis.
Anthology ID:
2026.findings-aacl.34
Volume:
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
645–682
Language:
URL:
https://aclanthology.org/2026.findings-aacl.34/
DOI:
Bibkey:
Cite (ACL):
Una Joh and Bei Yu. 2026. Beyond Top Words: MonoTM for Topic Modeling with Interpretable Monosemantic Features. In Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026, pages 645–682, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
Beyond Top Words: MonoTM for Topic Modeling with Interpretable Monosemantic Features (Joh & Yu, Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-aacl.34.pdf