Counterfactual Reasoning with Knowledge Graph Embeddings

Lena Zellinger, Andreas Stephan, Benjamin Roth


Abstract
Knowledge graph embeddings (KGEs) were originally developed to infer true but missing facts in incomplete knowledge repositories.In this paper, we link knowledge graph completion and counterfactual reasoning via our new task CFKGR. We model the original world state as a knowledge graph, hypothetical scenarios as edges added to the graph, and plausible changes to the graph as inferences from logical rules. We create corresponding benchmark datasets, which contain diverse hypothetical scenarios with plausible changes to the original knowledge graph and facts that should be retained. We develop COULDD, a general method for adapting existing knowledge graph embeddings given a hypothetical premise, and evaluate it on our benchmark. Our results indicate that KGEs learn patterns in the graph without explicit training. We further observe that KGEs adapted with COULDD solidly detect plausible counterfactual changes to the graph that follow these patterns. An evaluation on human-annotated data reveals that KGEs adapted with COULDD are mostly unable to recognize changes to the graph that do not follow learned inference rules. In contrast, ChatGPT mostly outperforms KGEs in detecting plausible changes to the graph but has poor knowledge retention. In summary, CFKGR connects two previously distinct areas, namely KG completion and counterfactual reasoning.
Anthology ID:
2024.eacl-long.168
Volume:
Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
March
Year:
2024
Address:
St. Julian’s, Malta
Editors:
Yvette Graham, Matthew Purver
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2753–2772
Language:
URL:
https://aclanthology.org/2024.eacl-long.168
DOI:
Bibkey:
Cite (ACL):
Lena Zellinger, Andreas Stephan, and Benjamin Roth. 2024. Counterfactual Reasoning with Knowledge Graph Embeddings. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2753–2772, St. Julian’s, Malta. Association for Computational Linguistics.
Cite (Informal):
Counterfactual Reasoning with Knowledge Graph Embeddings (Zellinger et al., EACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.eacl-long.168.pdf
Software:
 2024.eacl-long.168.software.zip
Note:
 2024.eacl-long.168.note.zip