More Than Words: Collocation Retokenization for Latent Dirichlet Allocation Models

Jin Cheevaprawatdomrong, Alexandra Schofield, Attapol Rutherford


Abstract
Traditionally, Latent Dirichlet Allocation (LDA) ingests words in a collection of documents to discover their latent topics using word-document co-occurrences. Previous studies show that representing bigrams collocations in the input can improve topic coherence in English. However, it is unclear how to achieve the best results for languages without marked word boundaries such as Chinese and Thai. Here, we explore the use of retokenization based on chi-squared measures, t-statistics, and raw frequency to merge frequent token ngrams into collocations when preparing input to the LDA model. Based on the goodness of fit and the coherence metric, we show that topics trained with merged tokens result in topic keys that are clearer, more coherent, and more effective at distinguishing topics than those of unmerged models.
Anthology ID:
2022.findings-acl.212
Volume:
Findings of the Association for Computational Linguistics: ACL 2022
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2696–2704
Language:
URL:
https://aclanthology.org/2022.findings-acl.212
DOI:
10.18653/v1/2022.findings-acl.212
Bibkey:
Cite (ACL):
Jin Cheevaprawatdomrong, Alexandra Schofield, and Attapol Rutherford. 2022. More Than Words: Collocation Retokenization for Latent Dirichlet Allocation Models. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2696–2704, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
More Than Words: Collocation Retokenization for Latent Dirichlet Allocation Models (Cheevaprawatdomrong et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-acl.212.pdf
Video:
 https://aclanthology.org/2022.findings-acl.212.mp4