Programmatic Tiered Prompting for LLM Generation of ELA & Math Items

David Whitecomb, Marjorie Wine, Alexander Hoffman


Abstract
Using the LBIDAT protocol, we evaluated items generated by three LLMs (Claude, Gemini, GPT) across six zero-shot prompting tiers for 8th-grade standards. Neither prompt tier nor model affected defect severity. ELA items were consistently poor; mathematics items passed the low bar while falling short of appropriate grade-level cognitive complexity.
Anthology ID:
2026.aimecon-sessions.29
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Coordinated Session Papers
Month:
October
Year:
2026
Address:
Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting-Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
267–273
Language:
URL:
https://aclanthology.org/2026.aimecon-sessions.29/
DOI:
Bibkey:
Cite (ACL):
David Whitecomb, Marjorie Wine, and Alexander Hoffman. 2026. Programmatic Tiered Prompting for LLM Generation of ELA & Math Items. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Coordinated Session Papers, pages 267–273, Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
Programmatic Tiered Prompting for LLM Generation of ELA & Math Items (Whitecomb et al., AIME-Con 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aimecon-sessions.29.pdf