A Cognitive Lab Study of Student AI Chatbot Use in Online Coursework

Jinah Choi, Sonya Powers, Farzan Karimi-Malekabadi, Michelle Barrett


Abstract
This cognitive lab study examined how secondary students in Grades 7-12 interacted with an AI-enabled chatbot while completing online course activities. Using chat logs, facilitator observations, post-session reflections, and survey responses, we analyzed spontaneous and lightly seeded interactions to evaluate whether the chatbot supported its intended theory of action: helping students continue learning without simply completing work for them. Findings highlight the importance of typing fluency, answer-boundary handling, safety and privacy response patterns, and student-facing feedback mechanisms. This study illustrates how cognitive lab evidence can inform ongoing validation.
Anthology ID:
2026.aimecon-wip.1
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress
Month:
October
Year:
2026
Address:
Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting-Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
1–8
Language:
URL:
https://aclanthology.org/2026.aimecon-wip.1/
DOI:
Bibkey:
Cite (ACL):
Jinah Choi, Sonya Powers, Farzan Karimi-Malekabadi, and Michelle Barrett. 2026. A Cognitive Lab Study of Student AI Chatbot Use in Online Coursework. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress, pages 1–8, Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
A Cognitive Lab Study of Student AI Chatbot Use in Online Coursework (Choi et al., AIME-Con 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aimecon-wip.1.pdf