Simulating Examinee Responses to Figure Matrices Items Using VLMs for Item Evaluation

Onur Demirkaya, Evelyn Johnson


Abstract
This study evaluated whether visual embeddings from SigLIP and DINOv2 could be used to simulate responses to Figure Matrices items. Models predicted response probabilities using item screenshots and examinee ability estimates. Results showed moderate probability recovery but limited item difficulty recovery, indicating promise for early screening but not calibration replacement.
Anthology ID:
2026.aimecon-main.56
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
Month:
October
Year:
2026
Address:
Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting-Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
497–505
Language:
URL:
https://aclanthology.org/2026.aimecon-main.56/
DOI:
Bibkey:
Cite (ACL):
Onur Demirkaya and Evelyn Johnson. 2026. Simulating Examinee Responses to Figure Matrices Items Using VLMs for Item Evaluation. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers, pages 497–505, Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
Simulating Examinee Responses to Figure Matrices Items Using VLMs for Item Evaluation (Demirkaya & Johnson, AIME-Con 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aimecon-main.56.pdf