Mark Dulhunty
Author directory2026
Beyond Agreement: Calibrating Automated Scoring to Human Judgment Using Control Scripts
Mark Dulhunty | Alex Codoreanu | Nathan Zoanetti
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress
Mark Dulhunty | Alex Codoreanu | Nathan Zoanetti
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress
An ensemble automated scoring approach was evaluated against 260,000 responses spanning 55 constructed-response reading items. Performance was then compared to 27 human markers using 477 control scripts. The ensemble achieved perfect agreement on control scripts for 39 items (71%), matching or exceeding human marker performance and supporting operational quality assurance.