English Machine Reading Comprehension Datasets: A Survey

Daria Dzendzik, Jennifer Foster, Carl Vogel


Abstract
This paper surveys 60 English Machine Reading Comprehension datasets, with a view to providing a convenient resource for other researchers interested in this problem. We categorize the datasets according to their question and answer form and compare them across various dimensions including size, vocabulary, data source, method of creation, human performance level, and first question word. Our analysis reveals that Wikipedia is by far the most common data source and that there is a relative lack of why, when, and where questions across datasets.
Anthology ID:
2021.emnlp-main.693
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8784–8804
Language:
URL:
https://aclanthology.org/2021.emnlp-main.693
DOI:
10.18653/v1/2021.emnlp-main.693
Bibkey:
Cite (ACL):
Daria Dzendzik, Jennifer Foster, and Carl Vogel. 2021. English Machine Reading Comprehension Datasets: A Survey. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8784–8804, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
English Machine Reading Comprehension Datasets: A Survey (Dzendzik et al., EMNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-main.693.pdf
Video:
 https://aclanthology.org/2021.emnlp-main.693.mp4
Code
 dariad/rczoo
Data
AmazonQABiPaRBookCorpusBoolQCliCRCoQACosmosQADROPDuoRCHotpotQAIIRCLAMBADAMCScriptMCTestMS MARCOMedHopMovieQAMultiRCNarrativeQANatural QuestionsNewsQAOpenBookQAQASCQUASARQUASAR-SQUASAR-TRACEReCOReClorReCoRDRecipeQASQuADSciQSearchQAShARCSubjQATriviaQATweetQAWho-did-WhatWikiHopWikiMoviesWikiQAWikiReadingWorldtreeemrQA