Kazunori Fukuhara
Author directory2026
Can LLMs Interpret Psychometric Parameters? Quantifying "Over-Knowledge" in LLM Student Simulation
Kazunori Fukuhara | Benjamin Domingue
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
Kazunori Fukuhara | Benjamin Domingue
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
LLM simulation suffers from an “over-knowledge problem”, where models perform too well to represent struggling learners. We compare persona-, IRT-, and CDM-based prompting for student simulation and measure how well methods reflect expected behaviors and quantify this bias. Findings show LLMs follow IRT parameters, yet struggle to simulate real abilities.