The development of autonomous web agents, powered by Large Language Models (LLMs) and reinforcement learning (RL), represents a significant step towards general-purpose AI assistants. However, training these agents is severely hampered by the challenges of interacting with the live internet, which is inefficient, costly, and fraught with risks. Model-based reinforcement learning (MBRL) offers a promising solution by learning a world model of the environment to enable simulated interaction. This paper introduces DynaWeb, a novel MBRL framework that trains web agents through interacting with a web world model trained to predict naturalistic web page representations given agent actions. This model serves as a synthetic web environment where an agent policy can dream by generating vast quantities of rollout action trajectories for efficient online reinforcement learning. Beyond free policy rollouts, DynaWeb incorporates real expert trajectories from training data, which are randomly interleaved with on-policy rollouts during training to improve stability and sample efficiency. Experiments conducted on the challenging WebArena and WebVoyager benchmarks demonstrate that DynaWeb consistently and significantly improves the performance of state-of-the-art open-source web agent models. Our findings establish the viability of training web agents through imagination, offering a scalable and efficient way to scale up online agentic RL.

Large language model (LLM)-based judges are increasingly used to scale the evaluation and optimization of open-ended medical dialogue systems, making their reliability critical in this high-stakes setting. However, existing medical benchmarks mainly assess their medical knowledge, while general judge benchmarks lack clinically grounded consultation criteria. To address this gap, we introduce Med-GRADE, a benchmark for evaluating LLMs as medical dialogue judges. It contains 2,076 real-world patient–LLM dialogues annotated by 56 physicians with a 23-item rubric adapted from Mini-CEX, covering Medical Interviewing (MI), Humanistic Care (HC), and Diagnosis and Treatment Management (DTM). Experiments on 17 LLMs show that current LLM judges achieve only moderate agreement, with the best model reaching 64.89% Macro-F1 and 68.28% Hamming Accuracy. Besides, more complex prompts may even harm performance: CoT disrupts parsing, while few-shot and self-refine introduce item-level instability. Data and code are available at https://github.com/jyd777/Med-GRADE for reproducibility.