CDM Q-Matrix Discovery with LLMs: Fusing Domain Knowledge with Empirical Evidence

Susu Zhang, V. N. Vimal Rao


Abstract
We present a method for diagnostic assessment Q-matrix specification combining LLM input with model-based empirical validation. An LLM generates an initial Q-matrix from item content, and response data guide subsequent Q refinement. The approach integrates substantive rationale with empirical evidence to support scalable and measurement theory-grounded diagnostic assessment.
Anthology ID:
2026.aimecon-wip.29
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress
Month:
October
Year:
2026
Address:
Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting-Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
216–230
Language:
URL:
https://aclanthology.org/2026.aimecon-wip.29/
DOI:
Bibkey:
Cite (ACL):
Susu Zhang and V. N. Vimal Rao. 2026. CDM Q-Matrix Discovery with LLMs: Fusing Domain Knowledge with Empirical Evidence. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress, pages 216–230, Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
CDM Q-Matrix Discovery with LLMs: Fusing Domain Knowledge with Empirical Evidence (Zhang & Rao, AIME-Con 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aimecon-wip.29.pdf