LLM-based multi-agent systems have demonstrated strong potential for solving complex tasks, yet reliable collaboration remains challenging. While most training-based methods rely on multi-agent trajectories, we investigate whether cooperative behaviors can be strengthened using only single-agent execution data. We propose CoopAgent, a training framework that learns solely from single-agent execution trajectories and transfers to multi-agent deployment. To bridge the gap between single-agent training and multi-agent environments, we introduce PADPO (Perturbation-Aligned DPO), which aligns action preferences between original observations and task-related perturbations that mimic changes caused by other agents. We train CoopAgent on single-agent execution trajectories collected in Minecraft and evaluate it on VillagerBench. The results demonstrate its potential to enhance collaboration on VillagerBench while preserving strong single-agent execution ability. We further assess CoopAgent on out-of-domain benchmarks, showing improvements in several multi-agent settings and performance comparable to the multi-agent training method AT-GRPO on the coding benchmarks. Our code is available at https://github.com/hlpzz/CoopAgent.

In this paper, we aim to evaluate multi-agent systems against complex dependencies, including spatial, causal, and temporal constraints. First, we construct a new benchmark, named VillagerBench, within the Minecraft environment. VillagerBench comprises diverse tasks crafted to test various aspects of multi-agent collaboration, from workload distribution to dynamic adaptation and synchronized task execution. Second, we introduce a Directed Acyclic Graph Multi-Agent Framework (VillagerAgent) to resolve complex inter-agent dependencies and enhance collaborative efficiency. This solution incorporates a task decomposer that creates a directed acyclic graph (DAG) for structured task management, an agent controller for task distribution, and a state manager for tracking environmental and agent data.Our empirical evaluation on VillagerBench demonstrates that VillagerAgentoutperforms the existing AgentVerse model, reducing hallucinations and improving task decomposition efficacy. The results underscore VillagerAgent’s potential in advancing multi-agent collaboration, offering a scalable and generalizable solution in dynamic environments. Source code is open-source on GitHub.