Chain-of-Thought Prompting for Automated Evaluation of Revision Patterns in Young Student Writing

Tianwen Li, Michelle Hong, Lindsay Clare Matsumura, Elaine Lin Wang, Diane Litman, Zhexiong Liu, Richard Correnti


Abstract
This study explores the use of ChatGPT-4.1 as a formative assessment tool for identifying revision patterns in young adolescents’ argumentative writing. ChatGPT-4.1 shows moderate agreement with human coders on identifying evidence-related revision patterns and fair agreement on explanation-related ones. Implications for LLM-assisted formative assessment of young adolescent writing are discussed.
Anthology ID:
2025.aimecon-wip.7
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress
Month:
October
Year:
2025
Address:
Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
49–65
Language:
URL:
https://aclanthology.org/2025.aimecon-wip.7/
DOI:
Bibkey:
Cite (ACL):
Tianwen Li, Michelle Hong, Lindsay Clare Matsumura, Elaine Lin Wang, Diane Litman, Zhexiong Liu, and Richard Correnti. 2025. Chain-of-Thought Prompting for Automated Evaluation of Revision Patterns in Young Student Writing. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress, pages 49–65, Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
Chain-of-Thought Prompting for Automated Evaluation of Revision Patterns in Young Student Writing (Li et al., AIME-Con 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.aimecon-wip.7.pdf