Automated Justification-Depth Scoring for Adaptive Support in AI-Supported Instructional Tasks

Songhee Han, Jueun Shin, Jiyoon Han


Abstract
This study evaluates machine-learning classifiers for detecting lower justification depth in student responses from AI-supported instructional design tasks. Using 600 human-coded responses and 20 group-aware repeated train-development-test partitions, RoBERTa outperformed TF-IDF logistic regression and Complement Naive Bayes under recall-prioritized thresholding for formative assessment support.
Anthology ID:
2026.aimecon-main.19
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
Month:
October
Year:
2026
Address:
Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting-Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
179–186
Language:
URL:
https://aclanthology.org/2026.aimecon-main.19/
DOI:
Bibkey:
Cite (ACL):
Songhee Han, Jueun Shin, and Jiyoon Han. 2026. Automated Justification-Depth Scoring for Adaptive Support in AI-Supported Instructional Tasks. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers, pages 179–186, Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
Automated Justification-Depth Scoring for Adaptive Support in AI-Supported Instructional Tasks (Han et al., AIME-Con 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aimecon-main.19.pdf