Evaluating an AI-Item Generation Tool for NAEP Reading

Cara Marie DiMeglio


Abstract
This study evaluated use of an AI-enabled item-generation tool for Reading item development for the National Assessment of Educational Progress (NAEP). Results were strongest when items targeted textually explicit information and weakest when passage and construct complexity increased. The tool showed potential for early-stage drafting rather than autonomous item development.
Anthology ID:
2026.aimecon-sessions.19
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Coordinated Session Papers
Month:
October
Year:
2026
Address:
Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting-Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
189–195
Language:
URL:
https://aclanthology.org/2026.aimecon-sessions.19/
DOI:
Bibkey:
Cite (ACL):
Cara Marie DiMeglio. 2026. Evaluating an AI-Item Generation Tool for NAEP Reading. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Coordinated Session Papers, pages 189–195, Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
Evaluating an AI-Item Generation Tool for NAEP Reading (DiMeglio, AIME-Con 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aimecon-sessions.19.pdf