Decomposing Co-occurrence Matrices into Interpretable Components as Formal Concepts

Akihiro Maeda, Takuma Torii, Shohei Hidaka


Abstract
This study addresses the interpretability of word representations through an investigation of a count-based co-occurrence matrix. Employing the mathematical methodology of Formal Concept Analysis, we reveal an underlying structure that is amenable to human interpretation. Furthermore, we unveil the emergence of hierarchical and geometrical structures within word vectors as consequences of word usage. Our experiments on the PPMI matrix demonstrate that the formal concepts that we identified align with interpretable categories, as shown in the category completion task.
Anthology ID:
2024.findings-acl.278
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4683–4700
Language:
URL:
https://aclanthology.org/2024.findings-acl.278
DOI:
Bibkey:
Cite (ACL):
Akihiro Maeda, Takuma Torii, and Shohei Hidaka. 2024. Decomposing Co-occurrence Matrices into Interpretable Components as Formal Concepts. In Findings of the Association for Computational Linguistics ACL 2024, pages 4683–4700, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
Decomposing Co-occurrence Matrices into Interpretable Components as Formal Concepts (Maeda et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-acl.278.pdf