LLM-Based Approaches for Detecting Gaming the System in Self-Explanation

Jiayi (Joyce) Zhang, Ryan S. Baker, Bruce M. McLaren


Abstract
This study compares two LLM-based approaches for detecting gaming behavior in students’ open-ended responses within a math digital learning game. The sentence embedding method outperformed the prompt-based approach and was more conservative. Consistent with prior research, gaming correlated negatively with learning, highlighting LLMs’ potential to detect disengagement in open-ended tasks.
Anthology ID:
2025.aimecon-main.10
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
Month:
October
Year:
2025
Address:
Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
91–98
Language:
URL:
https://aclanthology.org/2025.aimecon-main.10/
DOI:
Bibkey:
Cite (ACL):
Jiayi (Joyce) Zhang, Ryan S. Baker, and Bruce M. McLaren. 2025. LLM-Based Approaches for Detecting Gaming the System in Self-Explanation. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers, pages 91–98, Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
LLM-Based Approaches for Detecting Gaming the System in Self-Explanation (Zhang et al., AIME-Con 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.aimecon-main.10.pdf