Alleviating Sparsity of Open Knowledge Graphs with Ternary Contrastive Learning

Qian Li, Shafiq Joty, Daling Wang, Shi Feng, Yifei Zhang


Abstract
Sparsity of formal knowledge and roughness of non-ontological construction make sparsity problem particularly prominent in Open Knowledge Graphs (OpenKGs). Due to sparse links, learning effective representation for few-shot entities becomes difficult. We hypothesize that by introducing negative samples, a contrastive learning (CL) formulation could be beneficial in such scenarios. However, existing CL methods model KG triplets as binary objects of entities ignoring the relation-guided ternary propagation patterns and they are too generic, i.e., they ignore zero-shot, few-shot and synonymity problems that appear in OpenKGs. To address this, we propose TernaryCL, a CL framework based on ternary propagation patterns among head, relation and tail. TernaryCL designs Contrastive Entity and Contrastive Relation to mine ternary discriminative features with both negative entities and relations, introduces Contrastive Self to help zero- and few-shot entities learn discriminative features, Contrastive Synonym to model synonymous entities, and Contrastive Fusion to aggregate graph features from multiple paths. Extensive experiments on benchmarks demonstrate the superiority of TernaryCL over state-of-the-art models.
Anthology ID:
2022.findings-emnlp.168
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2022
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Editors:
Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2279–2291
Language:
URL:
https://aclanthology.org/2022.findings-emnlp.168
DOI:
10.18653/v1/2022.findings-emnlp.168
Bibkey:
Cite (ACL):
Qian Li, Shafiq Joty, Daling Wang, Shi Feng, and Yifei Zhang. 2022. Alleviating Sparsity of Open Knowledge Graphs with Ternary Contrastive Learning. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2279–2291, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
Alleviating Sparsity of Open Knowledge Graphs with Ternary Contrastive Learning (Li et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-emnlp.168.pdf
Video:
 https://aclanthology.org/2022.findings-emnlp.168.mp4