Evaluation of word embeddings against cognitive processes: primed reaction times in lexical decision and naming tasks

Jeremy Auguste, Arnaud Rey, Benoit Favre


Abstract
This work presents a framework for word similarity evaluation grounded on cognitive sciences experimental data. Word pair similarities are compared to reaction times of subjects in large scale lexical decision and naming tasks under semantic priming. Results show that GloVe embeddings lead to significantly higher correlation with experimental measurements than other controlled and off-the-shelf embeddings, and that the choice of a training corpus is less important than that of the algorithm. Comparison of rankings with other datasets shows that the cognitive phenomenon covers more aspects than simply word relatedness or similarity.
Anthology ID:
W17-5304
Volume:
Proceedings of the 2nd Workshop on Evaluating Vector Space Representations for NLP
Month:
September
Year:
2017
Address:
Copenhagen, Denmark
Editors:
Samuel Bowman, Yoav Goldberg, Felix Hill, Angeliki Lazaridou, Omer Levy, Roi Reichart, Anders Søgaard
Venue:
RepEval
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
21–26
Language:
URL:
https://aclanthology.org/W17-5304
DOI:
10.18653/v1/W17-5304
Bibkey:
Cite (ACL):
Jeremy Auguste, Arnaud Rey, and Benoit Favre. 2017. Evaluation of word embeddings against cognitive processes: primed reaction times in lexical decision and naming tasks. In Proceedings of the 2nd Workshop on Evaluating Vector Space Representations for NLP, pages 21–26, Copenhagen, Denmark. Association for Computational Linguistics.
Cite (Informal):
Evaluation of word embeddings against cognitive processes: primed reaction times in lexical decision and naming tasks (Auguste et al., RepEval 2017)
Copy Citation:
PDF:
https://aclanthology.org/W17-5304.pdf