Learning grounded word meaning representations on similarity graphs

Mariella Dimiccoli, Herwig Wendt, Pau Batlle Franch


Abstract
This paper introduces a novel approach to learn visually grounded meaning representations of words as low-dimensional node embeddings on an underlying graph hierarchy. The lower level of the hierarchy models modality-specific word representations, conditioned to another modality, through dedicated but communicating graphs, while the higher level puts these representations together on a single graph to learn a representation jointly from both modalities. The topology of each graph models similarity relations among words, and is estimated jointly with the graph embedding. The assumption underlying this model is that words sharing similar meaning correspond to communities in an underlying graph in a low-dimensional space. We named this model Hierarchical Multi-Modal Similarity Graph Embedding (HM-SGE). Experimental results validate the ability of HM-SGE to simulate human similarity judgments and concept categorization, outperforming the state of the art.
Anthology ID:
2021.emnlp-main.391
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4760–4769
Language:
URL:
https://aclanthology.org/2021.emnlp-main.391
DOI:
10.18653/v1/2021.emnlp-main.391
Bibkey:
Cite (ACL):
Mariella Dimiccoli, Herwig Wendt, and Pau Batlle Franch. 2021. Learning grounded word meaning representations on similarity graphs. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4760–4769, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Learning grounded word meaning representations on similarity graphs (Dimiccoli et al., EMNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-main.391.pdf
Video:
 https://aclanthology.org/2021.emnlp-main.391.mp4
Code
 mdimiccoli/hm-sge