Exploring Word Usage Change with Continuously Evolving Embeddings

Franziska Horn


Abstract
The usage of individual words can change over time, for example, when words experience a semantic shift. As text datasets generally comprise documents that were collected over a longer period of time, examining word usage changes in a corpus can often reveal interesting patterns. In this paper, we introduce a simple and intuitive way to track word usage changes via continuously evolving embeddings, computed as a weighted running average of transformer-based contextualized embeddings. We demonstrate our approach on a corpus of recent New York Times article snippets and provide code for an easy to use web app to conveniently explore semantic shifts with interactive plots.
Anthology ID:
2021.acl-demo.35
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations
Month:
August
Year:
2021
Address:
Online
Editors:
Heng Ji, Jong C. Park, Rui Xia
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
290–297
Language:
URL:
https://aclanthology.org/2021.acl-demo.35
DOI:
10.18653/v1/2021.acl-demo.35
Bibkey:
Cite (ACL):
Franziska Horn. 2021. Exploring Word Usage Change with Continuously Evolving Embeddings. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations, pages 290–297, Online. Association for Computational Linguistics.
Cite (Informal):
Exploring Word Usage Change with Continuously Evolving Embeddings (Horn, ACL-IJCNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-demo.35.pdf
Video:
 https://aclanthology.org/2021.acl-demo.35.mp4
Code
 cod3licious/evolvemb