Robert Margaryan
Author directory2026
MIMIC-CTRL-2026: A Realistic Clinical Trial Dataset to Benchmark LLM performance
Noshin Nawal | Robert Margaryan | Moustafa Abdalla | Justin A. Ezekowitz | Mohamed Abdalla
Findings of the Association for Computational Linguistics: EMNLP 2026
Noshin Nawal | Robert Margaryan | Moustafa Abdalla | Justin A. Ezekowitz | Mohamed Abdalla
Findings of the Association for Computational Linguistics: EMNLP 2026
Patient recruitment remains a major barrier in clinical trials, as identifying eligible participants often requires manual screening of complex health records, contributing significantly to trial delays and increased costs. While Large Language Models (LLMs) offer potential to automate this process, existing public datasets are overly simplistic in their patient representation: they pair full criterion lists with single-paragraph synthetic summaries, failing to capture the longitudinal complexity of real clinical records. We introduce MIMIC-CTRL-2026, a dataset of 800 manually labeled patient-criterion pairs grounded in real-world longitudinal EHR data from MIMIC-IV, annotated by clinically trained experts. Each instance pairs one eligibility criterion with a full patient record spanning multiple discharge notes, radiology reports, lab values, and medication histories. While simple prompting achieves high performance (κ = 0.84) on existing benchmarks, it collapses (κ = 0.06) on our dataset, revealing a substantial benchmark-difficulty gap between simplified and real-world clinical evaluation. Lastly, we present a framework combining structured pre-processing with a clinician-validated expanded prompt that substantially outperforms naive baselines without any model fine-tuning, establishing a realistic benchmark for automated clinical trial screening on real EHR data. Our dataset and code are available at: https://doi.org/10.5281/zenodo.22103011