BUCA: A Binary Classification Approach to Unsupervised Commonsense Question Answering

Jie He, Simon U, Victor Gutierrez-Basulto, Jeff Pan


Abstract
Unsupervised commonsense reasoning (UCR) is becoming increasingly popular as the construction of commonsense reasoning datasets is expensive, and they are inevitably limited in their scope. A popular approach to UCR is to fine-tune language models with external knowledge (e.g., knowledge graphs), but this usually requires a large number of training examples. In this paper, we propose to transform the downstream multiple choice question answering task into a simpler binary classification task by ranking all candidate answers according to their reasonableness. To this end, for training the model, we convert the knowledge graph triples into reasonable and unreasonable texts. Extensive experimental results show the effectiveness of our approach on various multiple choice question answering benchmarks. Furthermore, compared with existing UCR approaches using KGs, ours is less data hungry.
Anthology ID:
2023.acl-short.33
Volume:
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
376–387
Language:
URL:
https://aclanthology.org/2023.acl-short.33
DOI:
10.18653/v1/2023.acl-short.33
Bibkey:
Cite (ACL):
Jie He, Simon U, Victor Gutierrez-Basulto, and Jeff Pan. 2023. BUCA: A Binary Classification Approach to Unsupervised Commonsense Question Answering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 376–387, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
BUCA: A Binary Classification Approach to Unsupervised Commonsense Question Answering (He et al., ACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.acl-short.33.pdf
Video:
 https://aclanthology.org/2023.acl-short.33.mp4