A larger-scale evaluation resource of terms and their shift direction for diachronic lexical semantics

Astrid van Aggelen, Antske Fokkens, Laura Hollink, Jacco van Ossenbruggen


Abstract
Determining how words have changed their meaning is an important topic in Natural Language Processing. However, evaluations of methods to characterise such change have been limited to small, handcrafted resources. We introduce an English evaluation set which is larger, more varied, and more realistic than seen to date, with terms derived from a historical thesaurus. Moreover, the dataset is unique in that it represents change as a shift from the term of interest to a WordNet synset. Using the synset lemmas, we can use this set to evaluate (standard) methods that detect change between word pairs, as well as (adapted) methods that detect the change between a term and a sense overall. We show that performance on the new data set is much lower than earlier reported findings, setting a new standard.
Anthology ID:
W19-6105
Volume:
Proceedings of the 22nd Nordic Conference on Computational Linguistics
Month:
September–October
Year:
2019
Address:
Turku, Finland
Editors:
Mareike Hartmann, Barbara Plank
Venue:
NoDaLiDa
SIG:
Publisher:
Linköping University Electronic Press
Note:
Pages:
44–54
Language:
URL:
https://aclanthology.org/W19-6105
DOI:
Bibkey:
Cite (ACL):
Astrid van Aggelen, Antske Fokkens, Laura Hollink, and Jacco van Ossenbruggen. 2019. A larger-scale evaluation resource of terms and their shift direction for diachronic lexical semantics. In Proceedings of the 22nd Nordic Conference on Computational Linguistics, pages 44–54, Turku, Finland. Linköping University Electronic Press.
Cite (Informal):
A larger-scale evaluation resource of terms and their shift direction for diachronic lexical semantics (van Aggelen et al., NoDaLiDa 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-6105.pdf