Exploring Category Structure with Contextual Language Models and Lexical Semantic Networks

Joseph Renner, Pascal Denis, Remi Gilleron, Angèle Brunellière


Abstract
The psychological plausibility of word embeddings has been studied through different tasks such as word similarity, semantic priming, and lexical entailment. Recent work on predicting category structure with word embeddings report low correlations with human ratings. (Heyman and Heyman, 2019) showed that static word embeddings fail at predicting typicality using cosine similarity between category and exemplar words, while (Misra et al., 2021)obtain equally modest results for various contextual language models (CLMs) using a Cloze task formulation over hand-crafted taxonomic sentences. In this work, we test a wider array of methods for probing CLMs for predicting typicality scores. Our experiments, using BERT (Devlin et al., 2018), show the importance of using the right type of CLM probes, as our best BERT-based typicality prediction methods improve on previous works. Second, our results highlight the importance of polysemy in this task, as our best results are obtained when contextualization is paired with a disambiguation mechanism as in (Chronis and Erk, 2020). Finally, additional experiments and analyses reveal that Information Content-based WordNet (Miller, 1995) similarities with disambiguation match the performance of the best BERT-based method, and in fact capture complementary information, and when combined with BERT allow for enhanced typicality predictions.
Anthology ID:
2023.eacl-main.167
Volume:
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2277–2290
Language:
URL:
https://aclanthology.org/2023.eacl-main.167
DOI:
10.18653/v1/2023.eacl-main.167
Bibkey:
Cite (ACL):
Joseph Renner, Pascal Denis, Remi Gilleron, and Angèle Brunellière. 2023. Exploring Category Structure with Contextual Language Models and Lexical Semantic Networks. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2277–2290, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
Exploring Category Structure with Contextual Language Models and Lexical Semantic Networks (Renner et al., EACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.eacl-main.167.pdf
Dataset:
 2023.eacl-main.167.dataset.zip
Video:
 https://aclanthology.org/2023.eacl-main.167.mp4