Information-Theory Interpretation of the Skip-Gram Negative-Sampling Objective Function

Oren Melamud, Jacob Goldberger


Abstract
In this paper we define a measure of dependency between two random variables, based on the Jensen-Shannon (JS) divergence between their joint distribution and the product of their marginal distributions. Then, we show that word2vec’s skip-gram with negative sampling embedding algorithm finds the optimal low-dimensional approximation of this JS dependency measure between the words and their contexts. The gap between the optimal score and the low-dimensional approximation is demonstrated on a standard text corpus.
Anthology ID:
P17-2026
Volume:
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
July
Year:
2017
Address:
Vancouver, Canada
Editors:
Regina Barzilay, Min-Yen Kan
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
167–171
Language:
URL:
https://aclanthology.org/P17-2026
DOI:
10.18653/v1/P17-2026
Bibkey:
Cite (ACL):
Oren Melamud and Jacob Goldberger. 2017. Information-Theory Interpretation of the Skip-Gram Negative-Sampling Objective Function. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 167–171, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
Information-Theory Interpretation of the Skip-Gram Negative-Sampling Objective Function (Melamud & Goldberger, ACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/P17-2026.pdf
Video:
 https://aclanthology.org/P17-2026.mp4