Learning Geometric Word Meta-Embeddings

Pratik Jawanpuria, Satya Dev N T V, Anoop Kunchukuttan, Bamdev Mishra


Abstract
We propose a geometric framework for learning meta-embeddings of words from different embedding sources. Our framework transforms the embeddings into a common latent space, where, for example, simple averaging or concatenation of different embeddings (of a given word) is more amenable. The proposed latent space arises from two particular geometric transformations - source embedding specific orthogonal rotations and a common Mahalanobis metric scaling. Empirical results on several word similarity and word analogy benchmarks illustrate the efficacy of the proposed framework.
Anthology ID:
2020.repl4nlp-1.6
Volume:
Proceedings of the 5th Workshop on Representation Learning for NLP
Month:
July
Year:
2020
Address:
Online
Editors:
Spandana Gella, Johannes Welbl, Marek Rei, Fabio Petroni, Patrick Lewis, Emma Strubell, Minjoon Seo, Hannaneh Hajishirzi
Venue:
RepL4NLP
SIG:
SIGREP
Publisher:
Association for Computational Linguistics
Note:
Pages:
39–44
Language:
URL:
https://aclanthology.org/2020.repl4nlp-1.6
DOI:
10.18653/v1/2020.repl4nlp-1.6
Bibkey:
Cite (ACL):
Pratik Jawanpuria, Satya Dev N T V, Anoop Kunchukuttan, and Bamdev Mishra. 2020. Learning Geometric Word Meta-Embeddings. In Proceedings of the 5th Workshop on Representation Learning for NLP, pages 39–44, Online. Association for Computational Linguistics.
Cite (Informal):
Learning Geometric Word Meta-Embeddings (Jawanpuria et al., RepL4NLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.repl4nlp-1.6.pdf
Software:
 2020.repl4nlp-1.6.Software.zip
Video:
 http://slideslive.com/38929772