Jacqueline Church
Author directory2026
Responsible AI in the Duolingo English Test: Case Studies with Automatic Item Creation and Session-Level Quality Monitoring
Siyuan Marco Chen | Xiaowan Zhang | Andrew Runge | Jacqueline Church
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Coordinated Session Papers
Siyuan Marco Chen | Xiaowan Zhang | Andrew Runge | Jacqueline Church
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Coordinated Session Papers
Digital-first assessments are delivered continuously and often remotely. Artificial intelligence (AI) enables digital assessments to generate content and administer tests at scale. Like any system used for high-stakes decision-making, digital assessments require responsible AI (RAI) practices to ensure fairness and validity. This paper presents two deployed systems in the Duolingo English Test (DET) lifecycle that align the DET to its RAI Standards. The Item Factory combines automated item generation with staged expert review; the Analytics for Quality Assurance in Test Taker (AQUA-TT) system applies unsupervised anomaly detection methods to continuously monitor for issues in digital test deliveries for daily individual test sessions. We present the design and performance of these systems and discuss what they imply for placing human judgment inside digital assessments.
From Revision to Transfer: Longitudinal Effects of LLM Writing Feedback
Yigal Attali | Bryan Smith | Kai-Ling Lo | Andrew Runge | Jacqueline Church
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
Yigal Attali | Bryan Smith | Kai-Ling Lo | Andrew Runge | Jacqueline Church
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
Prior automated writing feedback systems have shown limited evidence of transferable writing development. This pilot longitudinal study examined whether repeated interaction with process-oriented LLM feedback improved independent writing performance over time. Results suggest positive transfer effects across novel writing tasks, supporting the potential developmental role of LLM-mediated feedback.