Read and Reason with MuSeRC and RuCoS: Datasets for Machine Reading Comprehension for Russian

Alena Fenogenova, Vladislav Mikhailov, Denis Shevelev


Abstract
The paper introduces two Russian machine reading comprehension (MRC) datasets, called MuSeRC and RuCoS, which require reasoning over multiple sentences and commonsense knowledge to infer the answer. The former follows the design of MultiRC, while the latter is a counterpart of the ReCoRD dataset. The datasets are included in RussianSuperGLUE, the Russian general language understanding benchmark. We provide a comparative analysis and demonstrate that the proposed tasks are relatively more complex as compared to the original ones for English. Besides, performance results of human solvers and BERT-based models show that MuSeRC and RuCoS represent a challenge for recent advanced neural models. We thus hope to facilitate research in the field of MRC for Russian and prompt the study of multi-hop reasoning in a cross-lingual scenario.
Anthology ID:
2020.coling-main.570
Volume:
Proceedings of the 28th International Conference on Computational Linguistics
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Donia Scott, Nuria Bel, Chengqing Zong
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
6481–6497
Language:
URL:
https://aclanthology.org/2020.coling-main.570
DOI:
10.18653/v1/2020.coling-main.570
Bibkey:
Cite (ACL):
Alena Fenogenova, Vladislav Mikhailov, and Denis Shevelev. 2020. Read and Reason with MuSeRC and RuCoS: Datasets for Machine Reading Comprehension for Russian. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6481–6497, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Cite (Informal):
Read and Reason with MuSeRC and RuCoS: Datasets for Machine Reading Comprehension for Russian (Fenogenova et al., COLING 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.coling-main.570.pdf
Data
ReCoRDSberQuADSuperGLUETaiga Corpus