Modeling Context Words as Regions: An Ordinal Regression Approach to Word Embedding

Shoaib Jameel, Steven Schockaert


Abstract
Vector representations of word meaning have found many applications in the field of natural language processing. Word vectors intuitively represent the average context in which a given word tends to occur, but they cannot explicitly model the diversity of these contexts. Although region representations of word meaning offer a natural alternative to word vectors, only few methods have been proposed that can effectively learn word regions. In this paper, we propose a new word embedding model which is based on SVM regression. We show that the underlying ranking interpretation of word contexts is sufficient to match, and sometimes outperform, the performance of popular methods such as Skip-gram. Furthermore, we show that by using a quadratic kernel, we can effectively learn word regions, which outperform existing unsupervised models for the task of hypernym detection.
Anthology ID:
K17-1014
Volume:
Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)
Month:
August
Year:
2017
Address:
Vancouver, Canada
Editors:
Roger Levy, Lucia Specia
Venue:
CoNLL
SIG:
SIGNLL
Publisher:
Association for Computational Linguistics
Note:
Pages:
123–133
Language:
URL:
https://aclanthology.org/K17-1014
DOI:
10.18653/v1/K17-1014
Bibkey:
Cite (ACL):
Shoaib Jameel and Steven Schockaert. 2017. Modeling Context Words as Regions: An Ordinal Regression Approach to Word Embedding. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 123–133, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
Modeling Context Words as Regions: An Ordinal Regression Approach to Word Embedding (Jameel & Schockaert, CoNLL 2017)
Copy Citation:
PDF:
https://aclanthology.org/K17-1014.pdf