Real-world clinical competence demands more than answering medical knowledge questions—it requires proactively gathering incomplete patient information, selecting appropriate examinations, and formulating safe, cost-conscious treatment plans through multi-turn interaction. Existing medical LLM benchmarks rely predominantly on static QA formats, failing to assess interactive information acquisition, tool use, or cost–quality tradeoffs. We propose MedCPEnv, a reproducible, cost-aware simulated hospital environment for evaluating and training medical LLM agents. Built from MIMIC-IV-NOTE discharge notes, MedCPEnv comprises 48,803 curated cases, a 1,000-case benchmark, and a 371,550-entry medical QA knowledge base, exposing four deterministic action types (ASK, EXAM, KNOWLEDGE, FINAL) to eliminate high-variance LLM role-playing artifacts. Evaluating eight frontier models, GPT-5.4 achieves the highest composite score (3.69/5) and diagnostic accuracy (60.3%), yet treatment accuracy remains only 19.4%, identifying treatment plan generation as the primary bottleneck. Substantial inter-model variation in the cost–quality tradeoff further demonstrates that clinical correctness alone is insufficient to characterize agent behavior. GRPO-based Agentic RL on Qwen3-4B shows that medical quality rewards consistently improve tool-use strategies, raising the composite score from 2.97 to 3.14 and diagnostic accuracy from 26.0% to 31.2%, establishing MedCPEnv as both a rigorous evaluation benchmark and a reproducible training environment for medical agent research.

Frontier large language models (LLMs) safety evaluation has largely treated harmful generation as an attack outcome rather than an object of analysis. Consequently, little is known about the outputs themselves, partly because large-scale, high-quality collections of frontier-model misbehavior are difficult to obtain. To address this gap, we introduce **HarmProfile**, a content-centric benchmark dataset that collects model misbehavior across diverse harm categories and model families, and defines the resulting content distribution as a model-level risk profile. The premise is that, just as linguistic behavior can be characterized from an utterance corpus, model risk can be characterized from the content, severity, and variation of its safety failures. HarmProfile contains over 80,000 validated artifacts from 23 frontier LLMs across 13 model families, organized into 15 harm categories and 57 subcategories. Using this corpus, we find that frontier LLMs reliably produces harmful content at scale, yet exhibit distinct risk profiles; both harmfulness and diversity grow with model capability, suggesting that frontier LLMs may appear safe yet harbor increasingly dangerous knowledge beneath the alignment surface. The code and dataset resources are publicly available at https://github.com/fresh-ma/HarmProfile.

Open-ended medical LLM evaluation remains weakly grounded in physician-calibrated coverage of clinically relevant response criteria, especially in localized clinical settings. We introduce ClinConsensus, a Chinese medical benchmark of 2,500 expert-curated cases spanning 36 specialties, 12 task themes, multiple difficulty levels, and lay-facing versus professional-facing settings. Each case is paired with 30 case-specific binary rubric criteria. To evaluate whether responses satisfy enough physician-authored criteria, we propose Clinician-Anchored Coverage Score (CACS), a physician-calibrated threshold metric instantiated at k = 10, and develop a dual-judge framework combining a GPT-5.1 grader with a physician-supervised Qwen3-8B judge. Evaluating 11 frontier LLMs, we find a persistent coverage gap: Rubric Accuracy ranges from 39.6% to 52.1%, whereas CACS@10 ranges from 17.8% to 32.9%, leaving a 19.2–21.9 point gap across models. Stratified analyses further reveal substantial variation across reasoning, evidence use, structured extraction, medication instructions, follow-up, and dialogue register. These results suggest that medical LLM evaluation should measure thresholded, rubric-grounded clinical coverage rather than average partial correctness.