Faithful Knowledge Graph Explanations in Commonsense Question Answering

Guy Aglionby, Simone Teufel


Abstract
Knowledge graphs are commonly used as sources of information in commonsense question answering, and can also be used to express explanations for the model’s answer choice. A common way of incorporating facts from the graph is to encode them separately from the question, and then combine the two representations to select an answer. In this paper, we argue that highly faithful graph-based explanations cannot be extracted from existing models of this type. Such explanations will not include reasoning done by the transformer encoding the question, so will be incomplete. We confirm this theory with a novel proxy measure for faithfulness and propose two architecture changes to address the problem. Our findings suggest a path forward for developing architectures for faithful graph-based explanations.
Anthology ID:
2022.emnlp-main.743
Volume:
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Editors:
Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
10811–10817
Language:
URL:
https://aclanthology.org/2022.emnlp-main.743
DOI:
10.18653/v1/2022.emnlp-main.743
Bibkey:
Cite (ACL):
Guy Aglionby and Simone Teufel. 2022. Faithful Knowledge Graph Explanations in Commonsense Question Answering. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10811–10817, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
Faithful Knowledge Graph Explanations in Commonsense Question Answering (Aglionby & Teufel, EMNLP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.emnlp-main.743.pdf