SentenceLDA: Discriminative and Robust Document Representation with Sentence Level Topic Model

Taehun Cha, Donghun Lee


Abstract
A subtle difference in context results in totally different nuances even for lexically identical words. On the other hand, two words can convey similar meanings given a homogeneous context. As a result, considering only word spelling information is not sufficient to obtain quality text representation. We propose SentenceLDA, a sentence-level topic model. We combine modern SentenceBERT and classical LDA to extend the semantic unit from word to sentence. By extending the semantic unit, we verify that SentenceLDA returns more discriminative document representation than other topic models, while maintaining LDA’s elegant probabilistic interpretability. We also verify the robustness of SentenceLDA by comparing the inference results on original and paraphrased texts. Additionally, we implement one possible application of SentenceLDA on corpus-level key opinion mining by applying SentenceLDA on an argumentative corpus, DebateSum.
Anthology ID:
2024.eacl-long.31
Volume:
Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
March
Year:
2024
Address:
St. Julian’s, Malta
Editors:
Yvette Graham, Matthew Purver
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
521–538
Language:
URL:
https://aclanthology.org/2024.eacl-long.31
DOI:
Bibkey:
Cite (ACL):
Taehun Cha and Donghun Lee. 2024. SentenceLDA: Discriminative and Robust Document Representation with Sentence Level Topic Model. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 521–538, St. Julian’s, Malta. Association for Computational Linguistics.
Cite (Informal):
SentenceLDA: Discriminative and Robust Document Representation with Sentence Level Topic Model (Cha & Lee, EACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.eacl-long.31.pdf
Software:
 2024.eacl-long.31.software.zip