RelWalk - A Latent Variable Model Approach to Knowledge Graph Embedding

Danushka Bollegala, Huda Hakami, Yuichi Yoshida, Ken-ichi Kawarabayashi


Abstract
Embedding entities and relations of a knowledge graph in a low-dimensional space has shown impressive performance in predicting missing links between entities. Although progresses have been achieved, existing methods are heuristically motivated and theoretical understanding of such embeddings is comparatively underdeveloped. This paper extends the random walk model of word embeddings to Knowledge Graph Embeddings (KGEs) to derive a scoring function that evaluates the strength of a relation R between two entities h (head) and t (tail). Moreover, we show that marginal loss minimisation, a popular objective used in much prior work in KGE, follows naturally from the log-likelihood ratio maximisation under the probabilities estimated from the KGEs according to our theoretical relationship. We propose a learning objective motivated by the theoretical analysis to learn KGEs from a given knowledge graph. Using the derived objective, accurate KGEs are learnt from FB15K237 and WN18RR benchmark datasets, providing empirical evidence in support of the theory.
Anthology ID:
2021.eacl-main.133
Volume:
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
Month:
April
Year:
2021
Address:
Online
Editors:
Paola Merlo, Jorg Tiedemann, Reut Tsarfaty
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1551–1565
Language:
URL:
https://aclanthology.org/2021.eacl-main.133
DOI:
10.18653/v1/2021.eacl-main.133
Bibkey:
Cite (ACL):
Danushka Bollegala, Huda Hakami, Yuichi Yoshida, and Ken-ichi Kawarabayashi. 2021. RelWalk - A Latent Variable Model Approach to Knowledge Graph Embedding. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1551–1565, Online. Association for Computational Linguistics.
Cite (Informal):
RelWalk - A Latent Variable Model Approach to Knowledge Graph Embedding (Bollegala et al., EACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.eacl-main.133.pdf
Data
FB15k-237