Cross-Topic Distributional Semantic Representations Via Unsupervised Mappings

Eleftheria Briakou, Nikos Athanasiou, Alexandros Potamianos


Abstract
In traditional Distributional Semantic Models (DSMs) the multiple senses of a polysemous word are conflated into a single vector space representation. In this work, we propose a DSM that learns multiple distributional representations of a word based on different topics. First, a separate DSM is trained for each topic and then each of the topic-based DSMs is aligned to a common vector space. Our unsupervised mapping approach is motivated by the hypothesis that words preserving their relative distances in different topic semantic sub-spaces constitute robust semantic anchors that define the mappings between them. Aligned cross-topic representations achieve state-of-the-art results for the task of contextual word similarity. Furthermore, evaluation on NLP downstream tasks shows that multiple topic-based embeddings outperform single-prototype models.
Anthology ID:
N19-1110
Volume:
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1052–1061
Language:
URL:
https://aclanthology.org/N19-1110
DOI:
10.18653/v1/N19-1110
Bibkey:
Copy Citation:
PDF:
https://aclanthology.org/N19-1110.pdf
Video:
 https://vimeo.com/355781410