Probing Commonsense Explanation in Dialogue Response Generation

Pei Zhou, Pegah Jandaghi, Hyundong Cho, Bill Yuchen Lin, Jay Pujara, Xiang Ren


Abstract
Humans use commonsense reasoning (CSR) implicitly to produce natural and coherent responses in conversations. Aiming to close the gap between current response generation (RG) models and human communication abilities, we want to understand why RG models respond as they do by probing RG model’s understanding of commonsense reasoning that elicits proper responses. We formalize the problem by framing commonsense as a latent variable in the RG task and using explanations for responses as textual form of commonsense. We collect 6k annotated explanations justifying responses from four dialogue datasets and ask humans to verify them and propose two probing settings to evaluate RG models’ CSR capabilities. Probing results show that models fail to capture the logical relations between commonsense explanations and responses and fine-tuning on in-domain data and increasing model sizes do not lead to understanding of CSR for RG. We hope our study motivates more research in making RG models emulate the human reasoning process in pursuit of smooth human-AI communication.
Anthology ID:
2021.findings-emnlp.349
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2021
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Venues:
EMNLP | Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
4132–4146
Language:
URL:
https://aclanthology.org/2021.findings-emnlp.349
DOI:
10.18653/v1/2021.findings-emnlp.349
Bibkey:
Cite (ACL):
Pei Zhou, Pegah Jandaghi, Hyundong Cho, Bill Yuchen Lin, Jay Pujara, and Xiang Ren. 2021. Probing Commonsense Explanation in Dialogue Response Generation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4132–4146, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Probing Commonsense Explanation in Dialogue Response Generation (Zhou et al., Findings 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.findings-emnlp.349.pdf
Data
GLUCOSEMuTualSocial IQATopical-Chat