From Expert Approval to Validity Evidence: Evaluating AI-Generated Assessment Items

Chris Meador


Abstract
Expert approval certifies that test developers would use AI-generated items, not that the scores support valid interpretations. I extend argument-based validity with a generation inference and a five-layer evidence framework, apply it diagnostically to three NAEP–Wilbur evaluations, and propose design principles and a minimal reporting standard for the field.
Anthology ID:
2026.aimecon-sessions.18
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Coordinated Session Papers
Month:
October
Year:
2026
Address:
Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting-Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
180–188
Language:
URL:
https://aclanthology.org/2026.aimecon-sessions.18/
DOI:
Bibkey:
Cite (ACL):
Chris Meador. 2026. From Expert Approval to Validity Evidence: Evaluating AI-Generated Assessment Items. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Coordinated Session Papers, pages 180–188, Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
From Expert Approval to Validity Evidence: Evaluating AI-Generated Assessment Items (Meador, AIME-Con 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aimecon-sessions.18.pdf