Yitong Qiao
Author directory2026
EHR-Complex: Benchmarking Medical Agents for Complex Clinical Reasoning
Yitong Qiao | Lei Liu | Yue Shen | Jian Wang | Jinjie Gu | Zhixuan Chu | Kui Ren
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yitong Qiao | Lei Liu | Yue Shen | Jian Wang | Jinjie Gu | Zhixuan Chu | Kui Ren
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Clinical agents promise to democratize access to electronic health records (EHRs), yet existing benchmarks fail to reflect the complexity of practical EHR analysis, e.g., often operating on idealized, clean EHRs via static SQL generation rather than interactive execution. In this work, we introduce **EHR-Complex**, a large-scale benchmark designed for interactive clinical database reasoning. Built on the large MIMIC-IV substrate (365K patients, 31 tables, 500M+ records), EHR-Complex comprises about 52K tasks spanning six clinical intents, supporting both patient-level and population-level queries, where each task requires an agent to interact with a sandboxed environment by executing SQL queries or Python code. Notably, EHR-Complex considers the real-world SQL task complexity for longitudinal multi-table aggregation and compositional reasoning, resulting in 31.93 SQL structural components per query on average. Evaluation results on EHR-Complex reveal the clinical difficulty of these EHR reasoning scenarios, with the top-performing model achieving only 62.3% exact-match accuracy. Pass^k consistency drops below 50% for nearly all evaluated models at k=4, exposing broad stochastic fragility. A fine-grained analysis of more than 3,800 failed trajectories for representative LLMs reveals three dominant failure modes: SQL logic errors, medical-code lookup failures, and semantic misunderstandings. EHR-Complex provides a rigorous testbed for clinical agents and highlights remaining gaps in robust reasoning for large-scale EHR analysis.
2025
Evaluating Robustness of Large Audio Language Models to Audio Injection: An Empirical Study
Guanyu Hou | Jiaming He | Yinhang Zhou | Ji Guo | Yitong Qiao | Rui Zhang | Wenbo Jiang
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Guanyu Hou | Jiaming He | Yinhang Zhou | Ji Guo | Yitong Qiao | Rui Zhang | Wenbo Jiang
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Large Audio-Language Models (LALMs) are increasingly deployed in real-world applications, yet their robustness against malicious audio injection remains underexplored. To address this gap, this study systematically evaluates five leading LALMs across four attack scenarios: Audio Interference Attack, Instruction Following Attack, Context Injection Attack, and Judgment Hijacking Attack. We quantitatively assess their vulnerabilities and resilience using metrics: the Defense Success Rate, Context Robustness Score, and Judgment Robustness Index. The experiments reveal significant performance disparities, with no single model demonstrating consistent robustness across all attack types. Attack effectiveness is significantly influenced by the position of the malicious content, particularly when injected at the beginning of a sequence. Furthermore, our analysis uncovers a negative correlation between a model’s instruction-following capability and its robustness: models that strictly adhere to instructions tend to be more susceptible, whereas safety-aligned models exhibit greater resistance. To facilitate future research, this work introduces a comprehensive benchmark framework. Our findings underscore the critical need for integrating robustness into training pipelines and developing multi-modal defenses, ultimately facilitating the secure deployment of LALMs. The dataset used in this work is available on Hugging Face.