Multimodal Item Parameter Estimation using Simulated Response Probabilities

Christopher Michael Ormerod, YoungKoung Kim


Abstract
We present results from reconstructing multiple-choice model (MCM) and three-parameter logistic (3PL) model curves using a fine-tuned multimodal large language model (LLM) based on Qwen3.5. The model is prompted and fine-tuned to replicate choice probabilities across a large training corpus of multiple-choice items containing both image and text stimuli, conditioned on a labeled set of student ability levels. By learning to reproduce the systematic error patterns of students across a discrete range of abilities, the LLM implicitly captures the underlying response probabilities encoded in the 3PL and MCM curves. This allows us to accurately approximate item difficulty on a held-out test set directly from the model’s predicted option probabilities.
Anthology ID:
2026.aimecon-main.9
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
Month:
October
Year:
2026
Address:
Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting-Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
83–91
Language:
URL:
https://aclanthology.org/2026.aimecon-main.9/
DOI:
Bibkey:
Cite (ACL):
Christopher Michael Ormerod and YoungKoung Kim. 2026. Multimodal Item Parameter Estimation using Simulated Response Probabilities. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers, pages 83–91, Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
Multimodal Item Parameter Estimation using Simulated Response Probabilities (Ormerod & Kim, AIME-Con 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aimecon-main.9.pdf