QAConv: Question Answering on Informative Conversations

Chien-Sheng Wu, Andrea Madotto, Wenhao Liu, Pascale Fung, Caiming Xiong


Abstract
This paper introduces QAConv, a new question answering (QA) dataset that uses conversations as a knowledge source. We focus on informative conversations, including business emails, panel discussions, and work channels. Unlike open-domain and task-oriented dialogues, these conversations are usually long, complex, asynchronous, and involve strong domain knowledge. In total, we collect 34,608 QA pairs from 10,259 selected conversations with both human-written and machine-generated questions. We use a question generator and a dialogue summarizer as auxiliary tools to collect and recommend questions. The dataset has two testing scenarios: chunk mode and full mode, depending on whether the grounded partial conversation is provided or retrieved. Experimental results show that state-of-the-art pretrained QA systems have limited zero-shot performance and tend to predict our questions as unanswerable. Our dataset provides a new training and evaluation testbed to facilitate QA on conversations research.
Anthology ID:
2022.acl-long.370
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5389–5411
Language:
URL:
https://aclanthology.org/2022.acl-long.370
DOI:
10.18653/v1/2022.acl-long.370
Bibkey:
Cite (ACL):
Chien-Sheng Wu, Andrea Madotto, Wenhao Liu, Pascale Fung, and Caiming Xiong. 2022. QAConv: Question Answering on Informative Conversations. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5389–5411, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
QAConv: Question Answering on Informative Conversations (Wu et al., ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-long.370.pdf
Software:
 2022.acl-long.370.software.zip
Video:
 https://aclanthology.org/2022.acl-long.370.mp4
Code
 salesforce/QAConv
Data
QAConvCoQADREAMMS MARCOMolweniQuACSQuAD