QUA-RC: the semi-synthetic dataset of multiple choice questions for assessing reading comprehension in Ukrainian

Mariia Zyrianova, Dmytro Kalpakchi


Abstract
In this article we present the first dataset of multiple choice questions for assessing reading comprehension in Ukrainian. The dataset is based on the texts from the Ukrainian national tests for reading comprehension, and the MCQs themselves are created semi-automatically in three stages. The first stage was to use GPT-3 to generate the MCQs zero-shot, the second stage was to select MCQs of sufficient quality and revise the ones with minor errors, whereas the final stage was to expand the dataset with the MCQs written manually. The dataset is created by the Ukrainian language native speakers, one of whom is also a language teacher. The resulting corpus has slightly more than 900 MCQs, of which only 43 MCQs could be kept as they were generated by GPT-3.
Anthology ID:
2023.nejlt-1.10
Volume:
Northern European Journal of Language Technology, Volume 9
Month:
Year:
2023
Address:
Linköping, Sweden
Editor:
Leon Derczynski
Venue:
NEJLT
SIG:
Publisher:
Linköping University Electronic Press
Note:
Pages:
Language:
URL:
https://aclanthology.org/2023.nejlt-1.10
DOI:
https://doi.org/10.3384/nejlt.2000-1533.2023.4939
Bibkey:
Cite (ACL):
Mariia Zyrianova and Dmytro Kalpakchi. 2023. QUA-RC: the semi-synthetic dataset of multiple choice questions for assessing reading comprehension in Ukrainian. In Northern European Journal of Language Technology, Volume 9, Linköping, Sweden. Linköping University Electronic Press.
Cite (Informal):
QUA-RC: the semi-synthetic dataset of multiple choice questions for assessing reading comprehension in Ukrainian (Zyrianova & Kalpakchi, NEJLT 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.nejlt-1.10.pdf