Syntactic Interchangeability in Word Embedding Models

Daniel Hershcovich, Assaf Toledo, Alon Halfon, Noam Slonim


Abstract
Nearest neighbors in word embedding models are commonly observed to be semantically similar, but the relations between them can vary greatly. We investigate the extent to which word embedding models preserve syntactic interchangeability, as reflected by distances between word vectors, and the effect of hyper-parameters—context window size in particular. We use part of speech (POS) as a proxy for syntactic interchangeability, as generally speaking, words with the same POS are syntactically valid in the same contexts. We also investigate the relationship between interchangeability and similarity as judged by commonly-used word similarity benchmarks, and correlate the result with the performance of word embedding models on these benchmarks. Our results will inform future research and applications in the selection of word embedding model, suggesting a principle for an appropriate selection of the context window size parameter depending on the use-case.
Anthology ID:
W19-2009
Volume:
Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP
Month:
June
Year:
2019
Address:
Minneapolis, USA
Editors:
Anna Rogers, Aleksandr Drozd, Anna Rumshisky, Yoav Goldberg
Venue:
RepEval
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
70–76
Language:
URL:
https://aclanthology.org/W19-2009
DOI:
10.18653/v1/W19-2009
Bibkey:
Cite (ACL):
Daniel Hershcovich, Assaf Toledo, Alon Halfon, and Noam Slonim. 2019. Syntactic Interchangeability in Word Embedding Models. In Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP, pages 70–76, Minneapolis, USA. Association for Computational Linguistics.
Cite (Informal):
Syntactic Interchangeability in Word Embedding Models (Hershcovich et al., RepEval 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-2009.pdf
Software:
 W19-2009.Software.zip
Poster:
 W19-2009.Poster.pdf
Code
 danielhers/interchangeability