Analysing Lexical Semantic Change with Contextualised Word Representations

Mario Giulianelli, Marco Del Tredici, Raquel Fernández


Abstract
This paper presents the first unsupervised approach to lexical semantic change that makes use of contextualised word representations. We propose a novel method that exploits the BERT neural language model to obtain representations of word usages, clusters these representations into usage types, and measures change along time with three proposed metrics. We create a new evaluation dataset and show that the model representations and the detected semantic shifts are positively correlated with human judgements. Our extensive qualitative analysis demonstrates that our method captures a variety of synchronic and diachronic linguistic phenomena. We expect our work to inspire further research in this direction.
Anthology ID:
2020.acl-main.365
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Editors:
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3960–3973
Language:
URL:
https://aclanthology.org/2020.acl-main.365
DOI:
10.18653/v1/2020.acl-main.365
Bibkey:
Cite (ACL):
Mario Giulianelli, Marco Del Tredici, and Raquel Fernández. 2020. Analysing Lexical Semantic Change with Contextualised Word Representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3960–3973, Online. Association for Computational Linguistics.
Cite (Informal):
Analysing Lexical Semantic Change with Contextualised Word Representations (Giulianelli et al., ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.365.pdf
Video:
 http://slideslive.com/38929048
Code
 glnmario/cwr4lsc