A Retrofitting Model for Incorporating Semantic Relations into Word Embeddings

Sapan Shah, Sreedhar Reddy, Pushpak Bhattacharyya


Abstract
We present a novel retrofitting model that can leverage relational knowledge available in a knowledge resource to improve word embeddings. The knowledge is captured in terms of relation inequality constraints that compare similarity of related and unrelated entities in the context of an anchor entity. These constraints are used as training data to learn a non-linear transformation function that maps original word vectors to a vector space respecting these constraints. The transformation function is learned in a similarity metric learning setting using Triplet network architecture. We applied our model to synonymy, antonymy and hypernymy relations in WordNet and observed large gains in performance over original distributional models as well as other retrofitting approaches on word similarity task and significant overall improvement on lexical entailment detection task.
Anthology ID:
2020.coling-main.111
Volume:
Proceedings of the 28th International Conference on Computational Linguistics
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Donia Scott, Nuria Bel, Chengqing Zong
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
1292–1298
Language:
URL:
https://aclanthology.org/2020.coling-main.111
DOI:
10.18653/v1/2020.coling-main.111
Bibkey:
Cite (ACL):
Sapan Shah, Sreedhar Reddy, and Pushpak Bhattacharyya. 2020. A Retrofitting Model for Incorporating Semantic Relations into Word Embeddings. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1292–1298, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Cite (Informal):
A Retrofitting Model for Incorporating Semantic Relations into Word Embeddings (Shah et al., COLING 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.coling-main.111.pdf