Automated Scoring of Clinical Expressive Language Evaluation Tasks

Yiyi Wang, Emily Prud’hommeaux, Meysam Asgari, Jill Dolata


Abstract
Many clinical assessment instruments used to diagnose language impairments in children include a task in which the subject must formulate a sentence to describe an image using a specific target word. Because producing sentences in this way requires the speaker to integrate syntactic and semantic knowledge in a complex manner, responses are typically evaluated on several different dimensions of appropriateness yielding a single composite score for each response. In this paper, we present a dataset consisting of non-clinically elicited responses for three related sentence formulation tasks, and we propose an approach for automatically evaluating their appropriateness. We use neural machine translation to generate correct-incorrect sentence pairs in order to create synthetic data to increase the amount and diversity of training data for our scoring model. Our scoring model uses transfer learning to facilitate automatic sentence appropriateness evaluation. We further compare custom word embeddings with pre-trained contextualized embeddings serving as features for our scoring model. We find that transfer learning improves scoring accuracy, particularly when using pretrained contextualized embeddings.
Anthology ID:
2020.bea-1.18
Volume:
Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications
Month:
July
Year:
2020
Address:
Seattle, WA, USA → Online
Editors:
Jill Burstein, Ekaterina Kochmar, Claudia Leacock, Nitin Madnani, Ildikó Pilán, Helen Yannakoudakis, Torsten Zesch
Venue:
BEA
SIG:
SIGEDU
Publisher:
Association for Computational Linguistics
Note:
Pages:
177–185
Language:
URL:
https://aclanthology.org/2020.bea-1.18
DOI:
10.18653/v1/2020.bea-1.18
Bibkey:
Cite (ACL):
Yiyi Wang, Emily Prud’hommeaux, Meysam Asgari, and Jill Dolata. 2020. Automated Scoring of Clinical Expressive Language Evaluation Tasks. In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 177–185, Seattle, WA, USA → Online. Association for Computational Linguistics.
Cite (Informal):
Automated Scoring of Clinical Expressive Language Evaluation Tasks (Wang et al., BEA 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.bea-1.18.pdf