ScAA: A Dataset for Automated Short Answer Grading of Children’s free-text Answers in Hindi and Marathi

Dolly Agarwal, Somya Gupta, Nishant Baghel


Abstract
Automatic short answer grading (ASAG) techniques are designed to automatically assess short answers written in natural language. Apart from MCQs, evaluating free text answer is essential to assess the knowledge and understanding of children in the subject. But assessing descriptive answers in low resource languages in a linguistically diverse country like India poses significant hurdles. To solve this assessment problem and advance NLP research in regional Indian languages, we present the Science Answer Assessment (ScAA) dataset of children’s answers in the age group of 8-14. ScAA dataset is a 2-way (correct/incorrect) labeled dataset and contains 10,988 and 1,955 pairs of natural answers along with model answers for Hindi and Marathi respectively for 32 questions. We benchmark various state-of-the-art ASAG methods, and show the data presents a strong challenge for future research.
Anthology ID:
2020.icon-main.58
Volume:
Proceedings of the 17th International Conference on Natural Language Processing (ICON)
Month:
December
Year:
2020
Address:
Indian Institute of Technology Patna, Patna, India
Editors:
Pushpak Bhattacharyya, Dipti Misra Sharma, Rajeev Sangal
Venue:
ICON
SIG:
Publisher:
NLP Association of India (NLPAI)
Note:
Pages:
430–436
Language:
URL:
https://aclanthology.org/2020.icon-main.58
DOI:
Bibkey:
Cite (ACL):
Dolly Agarwal, Somya Gupta, and Nishant Baghel. 2020. ScAA: A Dataset for Automated Short Answer Grading of Children’s free-text Answers in Hindi and Marathi. In Proceedings of the 17th International Conference on Natural Language Processing (ICON), pages 430–436, Indian Institute of Technology Patna, Patna, India. NLP Association of India (NLPAI).
Cite (Informal):
ScAA: A Dataset for Automated Short Answer Grading of Children’s free-text Answers in Hindi and Marathi (Agarwal et al., ICON 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.icon-main.58.pdf