Dinky Mishra

Author directory

2026

Simulated-patient benchmarks are widely used to evaluate clinical AI agents, yet the patient simulator itself can alter the evidence available to the agent and therefore the conclusions drawn from the benchmark. We introduce PULSE-CARE, a measurement-validity audit for simulated-patient clinical-agent evaluation that quantifies sensitivity to patient simulator behavior, agent architecture, simulator backbone, scorer choice, and demographic framing. Across 24 synthetic longitudinal cases, four dynamic simulator variants, three agent models, and two architectures, the mean per-case score range across simulators is 0.25 (95% CI: 0.21-0.29), and escalation decisions differ across simulator conditions for 60% of agent-case pairs. A 392-trajectory repeated-sampling analysis shows that simulator effects persist after pooling repeated draws, while also revealing substantial within-policy stochastic variation. Nominal model rank reversals occur in 17% of comparisons but disappear under a 0.05 practical-difference margin. Demographic counterfactuals change escalation in 26% of matched pairs, essentially the same rate as the same-input repeated-sampling control, providing no evidence of a demographic effect in this audit. PULSE-CARE combines simulator-sensitivity metrics, EFI composite indices, Patient Simulator Cards, and an Audit Report template to make the measurement conditions behind simulated-patient benchmark results explicit. Our results show that clinical-agent performance reported from a single patient simulator should be interpreted as conditional on the simulator and evaluation pipeline used to obtain it.