CoQA: A Conversational Question Answering Challenge

Siva Reddy, Danqi Chen, Christopher D. Manning


Abstract
Humans gather information through conversations involving a series of interconnected questions and answers. For machines to assist in information gathering, it is therefore essential to enable them to answer conversational questions. We introduce CoQA, a novel dataset for building Conversational Question Answering systems. Our dataset contains 127k questions with answers, obtained from 8k conversations about text passages from seven diverse domains. The questions are conversational, and the answers are free-form text with their corresponding evidence highlighted in the passage. We analyze CoQA in depth and show that conversational questions have challenging phenomena not present in existing reading comprehension datasets (e.g., coreference and pragmatic reasoning). We evaluate strong dialogue and reading comprehension models on CoQA. The best system obtains an F1 score of 65.4%, which is 23.4 points behind human performance (88.8%), indicating that there is ample room for improvement. We present CoQA as a challenge to the community at https://stanfordnlp.github.io/coqa.
Anthology ID:
Q19-1016
Volume:
Transactions of the Association for Computational Linguistics, Volume 7
Month:
Year:
2019
Address:
Cambridge, MA
Editors:
Lillian Lee, Mark Johnson, Brian Roark, Ani Nenkova
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
249–266
Language:
URL:
https://aclanthology.org/Q19-1016
DOI:
10.1162/tacl_a_00266
Bibkey:
Cite (ACL):
Siva Reddy, Danqi Chen, and Christopher D. Manning. 2019. CoQA: A Conversational Question Answering Challenge. Transactions of the Association for Computational Linguistics, 7:249–266.
Cite (Informal):
CoQA: A Conversational Question Answering Challenge (Reddy et al., TACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/Q19-1016.pdf
Code
 additional community code
Data
CoQAMCTestMS MARCONarrativeQASQuAD