Using AI-Simulated Response Probabilities for Item Difficulty Estimation

Brad Bolender, Sara Vispoel


Abstract
This paper evaluates an AI-based simulated administration approach for estimating item difficulty before empirical calibration. AI-generated option probabilities for synthetic examinees were converted into item difficulty estimates and compared with operational Rasch difficulty statistics. Results showed meaningful alignment, highlighting simulated administration as a potential tool for earlier item performance evidence.
Anthology ID:
2026.aimecon-main.68
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
Month:
October
Year:
2026
Address:
Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting-Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
606–611
Language:
URL:
https://aclanthology.org/2026.aimecon-main.68/
DOI:
Bibkey:
Cite (ACL):
Brad Bolender and Sara Vispoel. 2026. Using AI-Simulated Response Probabilities for Item Difficulty Estimation. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers, pages 606–611, Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
Using AI-Simulated Response Probabilities for Item Difficulty Estimation (Bolender & Vispoel, AIME-Con 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aimecon-main.68.pdf