Joint Semantic and Distributional Word Representations with Multi-Graph Embeddings

Pierre Daix-Moreux, Matthias Gallé


Abstract
Word embeddings continue to be of great use for NLP researchers and practitioners due to their training speed and easiness of use and distribution. Prior work has shown that the representation of those words can be improved by the use of semantic knowledge-bases. In this paper we propose a novel way of combining those knowledge-bases while the lexical information of co-occurrences of words remains. It is conceptually clear, as it consists in mapping both distributional and semantic information into a multi-graph and modifying existing node embeddings techniques to compute word representations. Our experiments show improved results compared to vanilla word embeddings, retrofitting and concatenation techniques using the same information, on a variety of data-sets of word similarities.
Anthology ID:
D19-5314
Volume:
Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13)
Month:
November
Year:
2019
Address:
Hong Kong
Venues:
EMNLP | TextGraphs | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
118–123
Language:
URL:
https://aclanthology.org/D19-5314
DOI:
10.18653/v1/D19-5314
Bibkey:
Cite (ACL):
Pierre Daix-Moreux and Matthias Gallé. 2019. Joint Semantic and Distributional Word Representations with Multi-Graph Embeddings. In Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13), pages 118–123, Hong Kong. Association for Computational Linguistics.
Cite (Informal):
Joint Semantic and Distributional Word Representations with Multi-Graph Embeddings (Daix-Moreux & Gallé, EMNLP 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-5314.pdf