Debiasing Event Understanding for Visual Commonsense Tasks

Minji Seo, YeonJoon Jung, Seungtaek Choi, Seung-won Hwang, Bei Liu


Abstract
We study event understanding as a critical step towards visual commonsense tasks. Meanwhile, we argue that current object-based event understanding is purely likelihood-based, leading to incorrect event prediction, due to biased correlation between events and objects. We propose to mitigate such biases with do-calculus, proposed in causality research, but overcoming its limited robustness, by an optimized aggregation with association-based prediction.We show the effectiveness of our approach, intrinsically by comparing our generated events with ground-truth event annotation, and extrinsically by downstream commonsense tasks.
Anthology ID:
2022.findings-acl.65
Volume:
Findings of the Association for Computational Linguistics: ACL 2022
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
782–787
Language:
URL:
https://aclanthology.org/2022.findings-acl.65
DOI:
10.18653/v1/2022.findings-acl.65
Bibkey:
Cite (ACL):
Minji Seo, YeonJoon Jung, Seungtaek Choi, Seung-won Hwang, and Bei Liu. 2022. Debiasing Event Understanding for Visual Commonsense Tasks. In Findings of the Association for Computational Linguistics: ACL 2022, pages 782–787, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Debiasing Event Understanding for Visual Commonsense Tasks (Seo et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-acl.65.pdf
Software:
 2022.findings-acl.65.software.zip
Data
VCR