Does an Automated Source Integration Score Respond to Revision? Extending Evidence of Construct Validity with a Feedback Experiment

Andrew Potter, Yu Tian, Kaitlin Van Houghton, Manmeet Singh, Renu Balyan, Maria Goldshtein, Laura K. Allen, Danielle S. McNamara


Abstract
This study examined whether an automated source integration measure responds to revision. In a randomized experiment, undergraduates drafted and revised a source-based essay. Source integration scores increased across drafts, although gains did not differ significantly between students who received source integration feedback and those who did not. Findings support use of the measure for formative evaluation and feedback.
Anthology ID:
2026.aimecon-wip.60
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress
Month:
October
Year:
2026
Address:
Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting-Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
483–490
Language:
URL:
https://aclanthology.org/2026.aimecon-wip.60/
DOI:
Bibkey:
Cite (ACL):
Andrew Potter, Yu Tian, Kaitlin Van Houghton, Manmeet Singh, Renu Balyan, Maria Goldshtein, Laura K. Allen, and Danielle S. McNamara. 2026. Does an Automated Source Integration Score Respond to Revision? Extending Evidence of Construct Validity with a Feedback Experiment. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress, pages 483–490, Wyndham Grand Pittsburgh Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
Does an Automated Source Integration Score Respond to Revision? Extending Evidence of Construct Validity with a Feedback Experiment (Potter et al., AIME-Con 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aimecon-wip.60.pdf