Alexey Zobnin


2019

pdf bib
Learning Word Embeddings without Context Vectors
Alexey Zobnin | Evgenia Elistratova
Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)

Most word embedding algorithms such as word2vec or fastText construct two sort of vectors: for words and for contexts. Naive use of vectors of only one sort leads to poor results. We suggest using indefinite inner product in skip-gram negative sampling algorithm. This allows us to use only one sort of vectors without loss of quality. Our “context-free” cf algorithm performs on par with SGNS on word similarity datasets