World models enable language agents to anticipate environment dynamics and evaluate candidate actions before execution, yet they are typically frozen after training, leaving them unable to adapt to the shifting on-policy distributions induced by evolving agents. Meanwhile, agent-improvement methods often depend on external rewards or verifiers that may be unavailable in realistic interactive environments. We introduce CoMap, a framework that co-evolves textual world models and agent policies through closed-loop interaction. At each decision step, the world model predicts future environment feedback for candidate actions, while the agent performs future-aware reflection by assessing the reliability of these predictions and refining its action accordingly. The resulting on-policy trajectories are then used to improve the world model through self-distillation, enabling it to continually adapt to the agent’s evolving interaction distribution. Across embodied task planning, web navigation, and tool-use benchmarks, CoMap consistently outperforms competitive baselines (e.g., a 16.75% relative improvement with Qwen3-4B). Further analyses show that this co-evolutionary process improves both world-model capability and long-horizon decision-making. Our code and data are available at: https://github.com/loyiv/CoMAP.

Recent advances in world models have shown promise for modeling future dynamics of environmental states, enabling agents to reason and act without accessing real environments. Current methods mainly perform single-step or fixed-horizon rollouts, leaving their potential for complex task planning under-exploited. We propose Imagine-then-Plan (ITP), a unified framework for agent learning via lookahead imagination, where an agent’s policy model interacts with the learned world model, yielding multi-step “imagined” trajectories. Since the imagination horizon may vary by tasks and stages, we introduce a novel adaptive lookahead mechanism by trading off the ultimate goal and task progress. The resulting imagined trajectories provide rich signals about future consequences, such as achieved progress and potential conflicts, which are fused with current observations, formulating a partially observable and imaginable Markov decision process to guide policy learning. We instantiate ITP with both training-free and reinforcement-trained variants. Extensive experiments across representative agent benchmarks demonstrate that ITP significantly outperforms competitive baselines. Further analyses validate that our adaptive lookahead largely enhances agents’ reasoning capability, providing valuable insights into addressing broader, complex tasks. Our code and data are publicly available at:https://github.com/loyiv/ITP.