Large language models (LLMs) have shown significant promise in healthcare applications. To better mirror real-world settings for LLM evaluation, dynamic longitudinal diagnosis-and-treatment simulation with virtual patients has recently emerged as a focal point of research. However, existing simulation frameworks are limited by clinical trajectory uniqueness, where virtual patients can only provide feedback based on information available in static electronic health records (EHRs). This limitation leads to simulation failures when unrecorded but medically sound examinations are ordered. In this paper, we formulate the task of Examination Feedback Simulation, which aims to dynamically augment the unique trajectory by simulating medically plausible examination results in response to clinical orders. To support this largely unexplored research area, we construct ClinTrack, a dataset curated from MIMIC-IV. We further propose a structure-aware evaluation metric SimScore to quantitatively assess the quality of simulated results, which shows promising initial alignment with expert judgment. Building on this framework, we develop ClinSim, a tailored generative model for this task. Experiments demonstrate that our 4B ClinSim model significantly outperforms flagship models with up to 235B parameters on this task, providing a critical foundation for creating more dynamic and realistic virtual patients. Code and other resources are available at GitHub.

In dialogue systems, role-playing agents (RPAs) aim to embody specific characters, but they often struggle to exhibit emotional expressions that are consistent with their assigned roles, resulting in insufficient emotional fidelity. This paper proposes EmoRL, a novel curriculum reinforcement learning (RL) framework, to enhance the emotional fidelity of RPAs. EmoRL features a multi-stage training paradigm guided by a carefully designed suite of reward functions. These rewards, including Format, Fine-grained Exact (FE), Emotional Similarity (ES), and Coarse-grained Categorical (CC) rewards, assess emotional consistency from strict to lenient perspectives. The curriculum RL process applies these rewards sequentially from easy (CC) to difficult (FE), enabling the RPAs to progressively learn coarse-grained emotional tendencies before mastering fine-grained distinctions. Furthermore, we introduce an emotional dialogue data synthesis pipeline to augment training data diversity and balance. Experiments on the Personality EmotionLines Dataset (PELD) demonstrate that EmoRL significantly outperforms baselines like RoleLLM, SFT, and DPO across both single-turn and multi-turn dialogues, with particularly notable gains in longer conversations. Our findings confirm the effectiveness of the curriculum learning strategy, the importance of balanced data, and the value of synthesized data for improving emotional fidelity.

As Large Language Models (LLMs) evolve into autonomous agents, the need for unified evaluation platform becomes critical. However, current evaluation pipelines remain highly fragmented and tightly coupled, hindering reproducibility and causing redundant engineering. To address this, we introduce AgentCompass, an open-source, lightweight, and extensible platform for evaluating LLM-based agents. AgentCompass organizes the evaluation process around three independent components, namely Benchmark, Harness, and Environment, thereby enabling flexible configurations without requiring the reimplementation of complex execution logic. Furthermore, it features a fault-tolerant asynchronous runtime and comprehensive trajectory analysis tools to transparently diagnose nuanced failure modes like reward-hacking. Natively supporting over 20 benchmarks across five capability dimensions, AgentCompass provides the community with a scalable and reproducible infrastructure for advancing agent research.

Role-playing agents (RPAs) powered by large language models (LLMs) have been widely utilized in dialogue systems for their capability to deliver personalized interactions. Current evaluations of RPAs mainly focus on personality fidelity, tone imitation, and knowledge consistency, while overlooking emotional fidelity, a key factor that affects user experience. To this end, we propose a benchmark called EmoCharacter to assess emotional fidelity of RPAs in dialogues. EmoCharacter includes two benchmark datasets (single-turn and multi-turn dialogues), three evaluation settings, and six metrics to measure the emotional fidelity between RPAs and the characters they portray. Based on EmoCharacter, we conduct extensive evaluations on RPAs powered by seven widely used LLMs with representative role-playing methods. Our empirical findings reveal that: (1) Contrary to intuition, current role-playing methods often reduce the emotional fidelity of LLMs in dialogues; (2) Enhancing the general capabilities of LLMs does not necessarily improve the emotional fidelity of RPAs; (3) Fine-tuning or In-Context Learning based on real dialogue data can enhance emotional fidelity.