ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers

Haitian Sun, William Cohen, Ruslan Salakhutdinov


Abstract
We describe a Question Answering (QA) dataset that contains complex questions with conditional answers, i.e. the answers are only applicable when certain conditions apply. We call this dataset ConditionalQA. In addition to conditional answers, the dataset also features:(1) long context documents with information that is related in logically complex ways;(2) multi-hop questions that require compositional logical reasoning;(3) a combination of extractive questions, yes/no questions, questions with multiple answers, and not-answerable questions;(4) questions asked without knowing the answers. We show that ConditionalQA is challenging for many of the existing QA models, especially in selecting answer conditions. We believe that this dataset will motivate further research in answering complex questions over long documents.
Anthology ID:
2022.acl-long.253
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3627–3637
Language:
URL:
https://aclanthology.org/2022.acl-long.253
DOI:
10.18653/v1/2022.acl-long.253
Bibkey:
Cite (ACL):
Haitian Sun, William Cohen, and Ruslan Salakhutdinov. 2022. ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3627–3637, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers (Sun et al., ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-long.253.pdf
Code
 haitian-sun/conditionalqa +  additional community code
Data
ConditionalQAPolicyQAQASPERShARC