TPPMI - a Temporal Positive Pointwise Mutual Information Embedding of Words

Paul Schmitt, Zsófia Rakovics, Márton Rakovics, Gábor Recski


Abstract
We present Temporal Positive Pointwise Mutual Information (TPPMI) embeddings as a robust and data-efficient alternative for modeling temporal semantic change. Based on the assumption that the semantics of the most frequent words in a corpus are relatively stable over time, our model represents words as vectors of their PPMI similarities with a predefined set of such context words. We evaluate our method on the temporal word analogy benchmark of Yao et al. (2018) and compare it to the TWEC model (Di Carlo et al., 2019), demonstrating the competitiveness of the approach. While the performance of TPPMI stays below that of the state-of-the-art TWEC model, it offers a higher degree of interpretability and is applicable in scenarios where only a limited amount of data is available.
Anthology ID:
2024.cpss-1.10
Volume:
Proceedings of the 4th Workshop on Computational Linguistics for the Political and Social Sciences: Long and short papers
Month:
Sep
Year:
2024
Address:
Vienna, Austria
Editors:
Christopher Klamm, Gabriella Lapesa, Simone Paolo Ponzetto, Ines Rehbein, Indira Sen
Venues:
cpss | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
119–125
Language:
URL:
https://aclanthology.org/2024.cpss-1.10
DOI:
Bibkey:
Cite (ACL):
Paul Schmitt, Zsófia Rakovics, Márton Rakovics, and Gábor Recski. 2024. TPPMI - a Temporal Positive Pointwise Mutual Information Embedding of Words. In Proceedings of the 4th Workshop on Computational Linguistics for the Political and Social Sciences: Long and short papers, pages 119–125, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
TPPMI - a Temporal Positive Pointwise Mutual Information Embedding of Words (Schmitt et al., cpss-WS 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.cpss-1.10.pdf