Explicit Retrofitting of Distributional Word Vectors

Goran Glavaš, Ivan Vulić


Abstract
Semantic specialization of distributional word vectors, referred to as retrofitting, is a process of fine-tuning word vectors using external lexical knowledge in order to better embed some semantic relation. Existing retrofitting models integrate linguistic constraints directly into learning objectives and, consequently, specialize only the vectors of words from the constraints. In this work, in contrast, we transform external lexico-semantic relations into training examples which we use to learn an explicit retrofitting model (ER). The ER model allows us to learn a global specialization function and specialize the vectors of words unobserved in the training data as well. We report large gains over original distributional vector spaces in (1) intrinsic word similarity evaluation and on (2) two downstream tasks − lexical simplification and dialog state tracking. Finally, we also successfully specialize vector spaces of new languages (i.e., unseen in the training data) by coupling ER with shared multilingual distributional vector spaces.
Anthology ID:
P18-1004
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
34–45
Language:
URL:
https://aclanthology.org/P18-1004
DOI:
10.18653/v1/P18-1004
Bibkey:
Cite (ACL):
Goran Glavaš and Ivan Vulić. 2018. Explicit Retrofitting of Distributional Word Vectors. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 34–45, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Explicit Retrofitting of Distributional Word Vectors (Glavaš & Vulić, ACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/P18-1004.pdf
Software:
 P18-1004.Software.zip
Presentation:
 P18-1004.Presentation.pdf
Video:
 https://vimeo.com/285807800