Dongsheng Shi

Author directory

2026

While recent medical benchmarks for Large Language Models (LLMs) have begun to incorporate clinical temporality, they still overlook a key reality: clinical evidence is encountered incrementally, requiring repeated decisions between reviewing the next available record and committing to a prediction. To address this issue, we introduce [<https://github.com/hwfengcs/INA-Bench>] , a benchmark for Inpatient Note Acquisition that evaluates LLMs’ ability to make acquire-or-predict decisions using sequentially revealed real-world Chinese inpatient records. In this benchmark, each case sequentially reveals up to three note blocks: an admission note, a first-course summary, and a daily-progress summary. After each block, the model under evaluation must either request the next block or predict one or more later documented follow-up actions (laboratory, imaging, pathology, or consultation). Surgery and discharge records are withheld for label construction and boundary checks only. is built from 8,986 de-identified inpatient records collected from two tertiary teaching hospitals, with patient-disjoint and temporal evaluation splits. Representative LLMs exhibit diverse acquire-or-predict behaviors. For example, DeepSeek-chat improves action micro-F1 by +0.102 and +0.074 on the patient-disjoint and temporal splits, respectively, while reading fewer note blocks, though the gains weaken under temporal drift.
Reliable clinical decision support requires tracking evidence over a hospital stay and revising a disease diagnosis when new findings challenge the initial assessment. Strong performance on medical question answering does not establish that large language models (LLMs) can make these longitudinal decisions. We introduce DiagBench, a benchmark of 2,865 Chinese inpatient admissions that evaluates diagnostic revision as clinical evidence accumulates. It pairs admission-only prediction with staged evidence disclosure, distinguishing an unchanged diagnosis, refinement of an existing hypothesis, and replacement by a different diagnosis. Development and test cases have human-annotated reference labels. The best LLM configuration in the main evaluation achieves only 52.6% macro recall; 80.3-87.7% of errors across five DeepSeek-Chat (DeepSeek-V3.2, non-thinking mode) prompting strategies occur between adjacent revision categories. Strategies that improve final-diagnosis prediction do not consistently improve revision decisions. We further use DiagAgent to trace diagnoses and evidence across rounds, revealing failures to replace an initial hypothesis despite recognizing contradictory findings. On DeepSeek-Chat, a single agent with the same state outperforms the full system on macro recall.
Large language model (LLM) driven multi-agent systems have shown promise in complex clinical reasoning, yet existing approaches rely on static strategies and lack persistent clinical memory, preventing self-evolving from prior diagnostic successes and failures. We present EMR, a self-evolving medical multi-agent system via Experience Mining and Reuse. EMR introduces a hierarchical clinical experience library that organizes accumulated knowledge into three levels: clinical principles, diagnostic patterns, and representative cases. During inference, EMR emulates multidisciplinary consultation: a planner agent coordinates domain-specific department agents for specialized reasoning, while a summary agent synthesizes their analyses into a final decision. Critically, EMR automatically extracts correct diagnostic insights and failure-related warnings from multi-agent reasoning trajectories, incrementally updating the experience library to guide future cases. Experiments on medical reasoning benchmarks demonstrate that EMR consistently outperforms state-of-the-art medical multi-agent baselines. Further analysis reveals that the hierarchical experience enables cross-specialty generalization and transfer across diverse LLM backbones, offering a scalable and interpretable pathway toward continually evolving intelligent medical reasoning.
Reliable model fingerprints are essential for protecting large language models (LLMs) against unauthorized redistribution and commercial misuse. In black-box deployment, verification is hindered by defensive filtering of suspected fingerprint queries, as well as by downstream model modifications that may weaken embedded ownership evidence. These risks require fingerprints to be robust in both construction and injection. For construction, prior paradigms face an imperceptibility trade-off: natural-language fingerprints may be accidentally activated, whereas garbled fingerprints are statistically exposed and easier to filter. For injection, existing methods struggle to preserve persistent trigger–target behaviors under model modification. We propose an end-to-end injected fingerprinting framework to address these challenges. Code-mixing Fingerprints (CF) use lowest-perplexity code-mixing under a high-complexity constraint to mitigate this two-sided imperceptibility trade-off. Multi-Candidate Editing (MCEdit) constructs structurally redundant, margin-separated trigger–target mappings to enable graceful degradation under model modification. Extensive evaluations on imperceptibility, detectability, and harmlessness demonstrate robust ownership verification with negligible impact on utility.

2025

With the increasing size of Large Vision-Language Models (LVLMs), network pruning techniques aimed at compressing models for deployment in resource-constrained environments have garnered significant attention. However, we observe that pruning often leads to a degradation in safety performance. To address this issue, we present a novel and lightweight approach, termed Hierarchical Safety Realignment (HSR). HSR operates by first quantifying the contribution of each attention head to safety, identifying the most critical ones, and then selectively restoring neurons directly within these attention heads that play a pivotal role in maintaining safety. This process hierarchically realigns the safety of pruned LVLMs, progressing from the attention head level to the neuron level. We validate HSR across various models and pruning strategies, consistently achieving notable improvements in safety performance. To our knowledge, this is the first work explicitly focused on restoring safety in LVLMs post-pruning.