The most plausible near-term role of medical LLMs is to assist rather than replace physicians, yet current evaluations often test isolated capabilities: clinical knowledge, EHR system interaction, or patient communication. Physician assistance instead requires coordinating these capabilities within the same interaction, where physicians issue underspecified requests, patients describe symptoms ambiguously, and EHR systems demand precise tool use. We introduce PhysAssistBench, a benchmark for interactive doctor-patient-EHR assistance. Built from real MIMIC-IV cases, PhysAssistBench uses a scalable pipeline to construct agentic patients: interactive, record-grounded agents that turn static EHR records into multi-turn clinical scenarios while preserving clinical factuality. PhysAssistBench provides a curated evaluation set of 1,296 manually reviewed and physician-validated turns. Experiments with leading LLMs show that current models remain unreliable in this setting, struggling to infer physician intent, clarify vague patient descriptions, formulate grounded EHR queries, and integrate evidence across sources. These results expose a key bottleneck for clinical LLMs: reliable assistance requires coordination across knowledge, communication, and systems, not isolated gains in any of them.

Medical VLMs frequently hallucinate clinically unsupported findings. We identify a stability gap: visually grounded tokens maintain relatively stable logits under semantics-preserving image perturbations, whereas visually misgrounded tokens exhibit substantially larger shifts. We introduce LENS (Logit-ENhanced Stability), which exploits this signal at both training and inference. During training, Semantics-Preserving Preference Optimization (SPPO) scores preference pairs using clinical faithfulness and cross-view stability; at inference, stability-aware contrastive decoding penalizes tokens whose logits diverge across original and counterfactual views. Constrained counterfactuals use attribution-guided masking and feature-space proximity guards to reduce disruption of lesion- and device-level evidence. Across radiology VQA, chest X-ray reporting, and gastrointestinal endoscopy, LENS consistently outperforms strong decoding-time and preference-optimization baselines. On MIMIC-CXR, entity-level RadGraph recall improves from 21.0 to 22.8 over PCCD, while expert evaluation shows a hallucination-rate reduction from 17.0% to 8.5%.