Language Based Mapping of Science Assessment Items to Skills

Farah Nadeem, Mari Ostendorf


Abstract
Knowledge of the association between assessment questions and the skills required to solve them is necessary for analysis of student learning. This association, often represented as a Q-matrix, is either hand-labeled by domain experts or learned as latent variables given a large student response data set. As a means of automating the match to formal standards, this paper uses neural text classification methods, leveraging the language in the standards documents to identify online text for a proxy training task. Experiments involve identifying the topic and crosscutting concepts of middle school science questions leveraging multi-task training. Results show that it is possible to automatically build a Q-matrix without student response data and using a modest number of hand-labeled questions.
Anthology ID:
W17-5036
Volume:
Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications
Month:
September
Year:
2017
Address:
Copenhagen, Denmark
Editors:
Joel Tetreault, Jill Burstein, Claudia Leacock, Helen Yannakoudakis
Venue:
BEA
SIG:
SIGEDU
Publisher:
Association for Computational Linguistics
Note:
Pages:
319–326
Language:
URL:
https://aclanthology.org/W17-5036
DOI:
10.18653/v1/W17-5036
Bibkey:
Cite (ACL):
Farah Nadeem and Mari Ostendorf. 2017. Language Based Mapping of Science Assessment Items to Skills. In Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications, pages 319–326, Copenhagen, Denmark. Association for Computational Linguistics.
Cite (Informal):
Language Based Mapping of Science Assessment Items to Skills (Nadeem & Ostendorf, BEA 2017)
Copy Citation:
PDF:
https://aclanthology.org/W17-5036.pdf
Attachment:
 W17-5036.Attachment.zip