Towards Automatic Bias Detection in Knowledge Graphs

Daphna Keidar, Mian Zhong, Ce Zhang, Yash Raj Shrestha, Bibek Paudel


Abstract
With the recent surge in social applications relying on knowledge graphs, the need for techniques to ensure fairness in KG based methods is becoming increasingly evident. Previous works have demonstrated that KGs are prone to various social biases, and have proposed multiple methods for debiasing them. However, in such studies, the focus has been on debiasing techniques, while the relations to be debiased are specified manually by the user. As manual specification is itself susceptible to human cognitive bias, there is a need for a system capable of quantifying and exposing biases, that can support more informed decisions on what to debias. To address this gap in the literature, we describe a framework for identifying biases present in knowledge graph embeddings, based on numerical bias metrics. We illustrate the framework with three different bias measures on the task of profession prediction, and it can be flexibly extended to further bias definitions and applications. The relations flagged as biased can then be handed to decision makers for judgement upon subsequent debiasing.
Anthology ID:
2021.findings-emnlp.321
Original:
2021.findings-emnlp.321v1
Version 2:
2021.findings-emnlp.321v2
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2021
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
3804–3811
Language:
URL:
https://aclanthology.org/2021.findings-emnlp.321
DOI:
10.18653/v1/2021.findings-emnlp.321
Bibkey:
Cite (ACL):
Daphna Keidar, Mian Zhong, Ce Zhang, Yash Raj Shrestha, and Bibek Paudel. 2021. Towards Automatic Bias Detection in Knowledge Graphs. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3804–3811, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Towards Automatic Bias Detection in Knowledge Graphs (Keidar et al., Findings 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.findings-emnlp.321.pdf
Video:
 https://aclanthology.org/2021.findings-emnlp.321.mp4
Code
 mianzg/kgbiasdetec