Examining decoding items using engine transcriptions and scoring in early literacy assessment

Zachary Schultz, Mackenzie Young, Debbie Dugdale, Susan Lottridge


Abstract
We investigate the reliability of two scoring approaches to early literacy decoding items, whereby students are shown a word and asked to say it aloud. Approaches were rubric scoring of speech, human or AI transcription with varying explicit scoring rules. Initial results suggest rubric-based approaches perform better than transcription-based methods.
Anthology ID:
2025.aimecon-wip.23
Volume:
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress
Month:
October
Year:
2025
Address:
Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States
Editors:
Joshua Wilson, Christopher Ormerod, Magdalen Beiting Parrish
Venue:
AIME-Con
SIG:
Publisher:
National Council on Measurement in Education (NCME)
Note:
Pages:
191–196
Language:
URL:
https://aclanthology.org/2025.aimecon-wip.23/
DOI:
Bibkey:
Cite (ACL):
Zachary Schultz, Mackenzie Young, Debbie Dugdale, and Susan Lottridge. 2025. Examining decoding items using engine transcriptions and scoring in early literacy assessment. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress, pages 191–196, Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME).
Cite (Informal):
Examining decoding items using engine transcriptions and scoring in early literacy assessment (Schultz et al., AIME-Con 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.aimecon-wip.23.pdf