Kevin Frome
2025
Automated Evaluation of Standardized Patients with LLMs
Andrew Emerson
|
Le An Ha
|
Keelan Evanini
|
Su Somay
|
Kevin Frome
|
Polina Harik
|
Victoria Yaneva
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
Standardized patients (SPs) are essential for clinical reasoning assessments in medical education. This paper introduces evaluation metrics that apply to both human and simulated SP systems. The metrics are computed using two LLM-as-a-judge approaches that align with human evaluators on SP performance, enabling scalable formative clinical reasoning assessments.
Search
Fix author
Co-authors
- Andrew Emerson 1
- Keelan Evanini 1
- Le An Ha 1
- Polina Harik 1
- Su Somay 1
- show all...