AI-Powered Coding of Elementary Students’ Small-Group Discussions about Text

Carla Firetto, P. Karen Murphy, Lin Yan, Yue Tang


Abstract
We report reliability and validity evidence for an AI-powered coding of 371 small-group discussion transcripts. Evidence via comparability and ground truth checks suggested high consistency between AI-produced and human-produced codes. Research in progress is also investigating reliability and validity of a new “quality” indicator to complement the current coding.
Anthology ID:
2025.aimecon-wip.15
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress
Month:
October
Year:
2025
Address:
Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
125–134
Language:
URL:
https://aclanthology.org/2025.aimecon-wip.15/
DOI:
Bibkey:
Cite (ACL):
Carla Firetto, P. Karen Murphy, Lin Yan, and Yue Tang. 2025. AI-Powered Coding of Elementary Students’ Small-Group Discussions about Text. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress, pages 125–134, Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
AI-Powered Coding of Elementary Students’ Small-Group Discussions about Text (Firetto et al., AIME-Con 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.aimecon-wip.15.pdf