ParaShoot: A Hebrew Question Answering Dataset

Omri Keren, Omer Levy


Abstract
NLP research in Hebrew has largely focused on morphology and syntax, where rich annotated datasets in the spirit of Universal Dependencies are available. Semantic datasets, however, are in short supply, hindering crucial advances in the development of NLP technology in Hebrew. In this work, we present ParaShoot, the first question answering dataset in modern Hebrew. The dataset follows the format and crowdsourcing methodology of SQuAD, and contains approximately 3000 annotated examples, similar to other question-answering datasets in low-resource languages. We provide the first baseline results using recently-released BERT-style models for Hebrew, showing that there is significant room for improvement on this task.
Anthology ID:
2021.mrqa-1.11
Volume:
Proceedings of the 3rd Workshop on Machine Reading for Question Answering
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Editors:
Adam Fisch, Alon Talmor, Danqi Chen, Eunsol Choi, Minjoon Seo, Patrick Lewis, Robin Jia, Sewon Min
Venue:
MRQA
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
106–112
Language:
URL:
https://aclanthology.org/2021.mrqa-1.11
DOI:
10.18653/v1/2021.mrqa-1.11
Bibkey:
Cite (ACL):
Omri Keren and Omer Levy. 2021. ParaShoot: A Hebrew Question Answering Dataset. In Proceedings of the 3rd Workshop on Machine Reading for Question Answering, pages 106–112, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
ParaShoot: A Hebrew Question Answering Dataset (Keren & Levy, MRQA 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.mrqa-1.11.pdf
Code
 omrikeren/parashoot
Data
ParaShootOSCARSQuAD