LLM-Based Rubric Refinement in Multi-Agent Automated Scoring

Alexander Kwako, Cristina Everett, Harry Wang


Abstract
In this study, we marry automated scoring with automated rubric revision in an iterative, mutually-reinforcing process. One LLM agent scores student responses; a second, the Error Analysis Agent, examines human–engine scoring discrepancies and suggests rubric revisions. We show that this positive feedback loop improves automated scoring performance.
Anthology ID:
2026.aimecon-sessions.2
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Coordinated Session Papers
Month:
October
Year:
2026
Address:
Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting-Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
9–22
Language:
URL:
https://aclanthology.org/2026.aimecon-sessions.2/
DOI:
Bibkey:
Cite (ACL):
Alexander Kwako, Cristina Everett, and Harry Wang. 2026. LLM-Based Rubric Refinement in Multi-Agent Automated Scoring. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Coordinated Session Papers, pages 9–22, Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
LLM-Based Rubric Refinement in Multi-Agent Automated Scoring (Kwako et al., AIME-Con 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aimecon-sessions.2.pdf