Beyond Clean Agreement: A Stability-Bounded Evaluation of Compact AES Distillation

Yi Gui


Abstract
Compact distillation preserved clean essay-scoring agreement across three student models and six compressed teacher targets, but the best target differed by model and margins were small. Perturbation, distribution-shift, and subgroup-bootstrap analyses dominated those differences, indicating that resource-constrained educational scoring requires uncertainty-aware evaluation rather than clean held-out comparison alone.
Anthology ID:
2026.aimecon-sessions.5
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Coordinated Session Papers
Month:
October
Year:
2026
Address:
Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting-Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
39–48
Language:
URL:
https://aclanthology.org/2026.aimecon-sessions.5/
DOI:
Bibkey:
Cite (ACL):
Yi Gui. 2026. Beyond Clean Agreement: A Stability-Bounded Evaluation of Compact AES Distillation. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Coordinated Session Papers, pages 39–48, Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
Beyond Clean Agreement: A Stability-Bounded Evaluation of Compact AES Distillation (Gui, AIME-Con 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aimecon-sessions.5.pdf