A Greedy Bit-flip Training Algorithm for Binarized Knowledge Graph Embeddings

Katsuhiko Hayashi, Koki Kishimoto, Masashi Shimbo


Abstract
This paper presents a simple and effective discrete optimization method for training binarized knowledge graph embedding model B-CP. Unlike the prior work using a SGD-based method and quantization of real-valued vectors, the proposed method directly optimizes binary embedding vectors by a series of bit flipping operations. On the standard knowledge graph completion tasks, the B-CP model trained with the proposed method achieved comparable performance with that trained with SGD as well as state-of-the-art real-valued models with similar embedding dimensions.
Anthology ID:
2020.findings-emnlp.10
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2020
Month:
November
Year:
2020
Address:
Online
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
109–114
Language:
URL:
https://aclanthology.org/2020.findings-emnlp.10
DOI:
10.18653/v1/2020.findings-emnlp.10
Bibkey:
Cite (ACL):
Katsuhiko Hayashi, Koki Kishimoto, and Masashi Shimbo. 2020. A Greedy Bit-flip Training Algorithm for Binarized Knowledge Graph Embeddings. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 109–114, Online. Association for Computational Linguistics.
Cite (Informal):
A Greedy Bit-flip Training Algorithm for Binarized Knowledge Graph Embeddings (Hayashi et al., Findings 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.findings-emnlp.10.pdf
Data
FB15k-237