Incorporating Subword Information into Matrix Factorization Word Embeddings

Alexandre Salle, Aline Villavicencio


Abstract
The positive effect of adding subword information to word embeddings has been demonstrated for predictive models. In this paper we investigate whether similar benefits can also be derived from incorporating subwords into counting models. We evaluate the impact of different types of subwords (n-grams and unsupervised morphemes), with results confirming the importance of subword information in learning representations of rare and out-of-vocabulary words.
Anthology ID:
W18-1209
Volume:
Proceedings of the Second Workshop on Subword/Character LEvel Models
Month:
June
Year:
2018
Address:
New Orleans
Editors:
Manaal Faruqui, Hinrich Schütze, Isabel Trancoso, Yulia Tsvetkov, Yadollah Yaghoobzadeh
Venue:
SCLeM
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
66–71
Language:
URL:
https://aclanthology.org/W18-1209
DOI:
10.18653/v1/W18-1209
Bibkey:
Cite (ACL):
Alexandre Salle and Aline Villavicencio. 2018. Incorporating Subword Information into Matrix Factorization Word Embeddings. In Proceedings of the Second Workshop on Subword/Character LEvel Models, pages 66–71, New Orleans. Association for Computational Linguistics.
Cite (Informal):
Incorporating Subword Information into Matrix Factorization Word Embeddings (Salle & Villavicencio, SCLeM 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-1209.pdf
Video:
 https://aclanthology.org/W18-1209.mp4
Code
 alexandres/lexvec