Alexandra Perez
Author directory2026
Predicting Item-to-Range Performance Level Descriptor Matches with Structured Language Model Reasoning
Kang Xue | M. Christina Schneider | Sangdon Lim | Garron Gianopulos | Alexandra Perez
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress
Kang Xue | M. Christina Schneider | Sangdon Lim | Garron Gianopulos | Alexandra Perez
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress
Range performance level descriptors (RPLDs) connect assessment items to claims about what students at different achievement levels know and can do. Retrospectively assigning RPLDs to a large item bank is valuable but labor intensive. This work-in-progress study evaluates whether small and large language models can assist expert item-to-RPLD matching for 524 Grade 3–5 English language arts items. We compared direct classification with structured prompts that guide a model to analyze the knowledge, skills, evidence, and cognitive demand required by an item. We also examined self-consistency voting and a rater-informed prompting. Preliminary results indicate that larger hosted models achieved the closest overall performance to humans, although some locally hosted models produced comparable results. Voting improved prediction reliability but did not consistently increase agreement with human scores, whereas training models with human scores generally improved classification accuracy. Model performance also tended to decline as grade level increased.