Evaluating Word Embeddings in Multi-label Classification Using Fine-Grained Name Typing

Yadollah Yaghoobzadeh, Katharina Kann, Hinrich Schütze


Abstract
Embedding models typically associate each word with a single real-valued vector, representing its different properties. Evaluation methods, therefore, need to analyze the accuracy and completeness of these properties in embeddings. This requires fine-grained analysis of embedding subspaces. Multi-label classification is an appropriate way to do so. We propose a new evaluation method for word embeddings based on multi-label classification given a word embedding. The task we use is fine-grained name typing: given a large corpus, find all types that a name can refer to based on the name embedding. Given the scale of entities in knowledge bases, we can build datasets for this task that are complementary to the current embedding evaluation datasets in: they are very large, contain fine-grained classes, and allow the direct evaluation of embeddings without confounding factors like sentence context.
Anthology ID:
W18-3013
Volume:
Proceedings of the Third Workshop on Representation Learning for NLP
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Isabelle Augenstein, Kris Cao, He He, Felix Hill, Spandana Gella, Jamie Kiros, Hongyuan Mei, Dipendra Misra
Venue:
RepL4NLP
SIG:
SIGREP
Publisher:
Association for Computational Linguistics
Note:
Pages:
101–106
Language:
URL:
https://aclanthology.org/W18-3013
DOI:
10.18653/v1/W18-3013
Bibkey:
Cite (ACL):
Yadollah Yaghoobzadeh, Katharina Kann, and Hinrich Schütze. 2018. Evaluating Word Embeddings in Multi-label Classification Using Fine-Grained Name Typing. In Proceedings of the Third Workshop on Representation Learning for NLP, pages 101–106, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Evaluating Word Embeddings in Multi-label Classification Using Fine-Grained Name Typing (Yaghoobzadeh et al., RepL4NLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-3013.pdf
Code
 yyaghoobzadeh/name_typing