GRI: Graph-based Relative Isomorphism of Word Embedding Spaces

Muhammad Ali, Yan Hu, Jianbin Qin, Di Wang


Abstract
Automated construction of bi-lingual dictionaries using monolingual embedding spaces is a core challenge in machine translation. The end performance of these dictionaries relies on the geometric similarity of individual spaces, i.e., their degree of isomorphism. Existing attempts aimed at controlling the relative isomorphism of different spaces fail to incorporate the impact of lexically different but semantically related words in the training objective. To address this, we propose GRI that combines the distributional training objectives with attentive graph convolutions to unanimously consider the impact of lexical variations of semantically similar words required to define/compute the relative isomorphism of multiple spaces. Exper imental evaluation shows that GRI outperforms the existing research by improving the average P@1 by a relative score of upto 63.6%.
Anthology ID:
2023.findings-emnlp.756
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2023
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
11304–11313
Language:
URL:
https://aclanthology.org/2023.findings-emnlp.756
DOI:
10.18653/v1/2023.findings-emnlp.756
Bibkey:
Cite (ACL):
Muhammad Ali, Yan Hu, Jianbin Qin, and Di Wang. 2023. GRI: Graph-based Relative Isomorphism of Word Embedding Spaces. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11304–11313, Singapore. Association for Computational Linguistics.
Cite (Informal):
GRI: Graph-based Relative Isomorphism of Word Embedding Spaces (Ali et al., Findings 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.findings-emnlp.756.pdf