Su Somay


2025

pdf bib
Automated Evaluation of Standardized Patients with LLMs
Andrew Emerson | Le An Ha | Keelan Evanini | Su Somay | Kevin Frome | Polina Harik | Victoria Yaneva
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers

Standardized patients (SPs) are essential for clinical reasoning assessments in medical education. This paper introduces evaluation metrics that apply to both human and simulated SP systems. The metrics are computed using two LLM-as-a-judge approaches that align with human evaluators on SP performance, enabling scalable formative clinical reasoning assessments.