Are Embedded Potatoes Still Vegetables? On the Limitations of WordNet Embeddings for Lexical Semantics

Xuyou Cheng, Michael Schlichtkrull, Guy Emerson


Abstract
Knowledge Base Embedding (KBE) models have been widely used to encode structured information from knowledge bases, including WordNet. However, the existing literature has predominantly focused on link prediction as the evaluation task, often neglecting exploration of the models’ semantic capabilities. In this paper, we investigate the potential disconnect between the performance of KBE models of WordNet on link prediction and their ability to encode semantic information, highlighting the limitations of current evaluation protocols. Our findings reveal that some top-performing KBE models on the WN18RR benchmark exhibit subpar results on two semantic tasks and two downstream tasks. These results demonstrate the inadequacy of link prediction benchmarks for evaluating the semantic capabilities of KBE models, suggesting the need for a more targeted assessment approach.
Anthology ID:
2023.emnlp-main.542
Volume:
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8763–8775
Language:
URL:
https://aclanthology.org/2023.emnlp-main.542
DOI:
10.18653/v1/2023.emnlp-main.542
Bibkey:
Cite (ACL):
Xuyou Cheng, Michael Schlichtkrull, and Guy Emerson. 2023. Are Embedded Potatoes Still Vegetables? On the Limitations of WordNet Embeddings for Lexical Semantics. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8763–8775, Singapore. Association for Computational Linguistics.
Cite (Informal):
Are Embedded Potatoes Still Vegetables? On the Limitations of WordNet Embeddings for Lexical Semantics (Cheng et al., EMNLP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.emnlp-main.542.pdf
Video:
 https://aclanthology.org/2023.emnlp-main.542.mp4