Yiting Yao
Author directory2026
An Evaluation of Agreement and Uncertainty in LLM-Based Automated Essay Scoring
Yiting Yao | Yanyun Yang | Huan (Hailey) Kuang
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
Yiting Yao | Yanyun Yang | Huan (Hailey) Kuang
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
This study evaluates score agreement and uncertainty estimation in large language model (LLM)-based automated essay scoring. Results showed that LLMs tended to be overconfident and highly consistent in their assigned scores, and majority voting did not yield stronger agreement with human scores than deterministic scoring.