Large language model (LLM)-based multi-agent systems (MASs) have shown impressive performance in solving a wide range of complex problems. However, previous studies mainly focus on designing customized MAS for specific tasks, while a critical research problem remains unclear: Do LLM agent groups exhibit a form of “general intelligence” that reflects their general ability across various tasks? Researchers have found a Collective Intelligence (CI) factor in human groups that captures their general capability. Inspired by this, in this study, we aim to investigate whether an analogous CI factor also exists in LLM agent groups, which is crucial for building generalizable MAS. Motivated by human cognitive psychology experiments, we construct 108 LLM agent groups with diverse group sizes, LLM compositions, and communication topologies. We systematically evaluate these groups across a wide range of tasks and analyze their performances. Our results demonstrate that an Artificial Collective Intelligence (ACI) factor can be extracted from LLM agent groups to predict the generalization performance on new tasks. Inspired by this, we train a model to predict the ACI based on the features of MAS, and show that it can be used as a plug-in to enhance the generalization ability of MAS optimization methods.

Large Language Model (LLM) agents are increasingly expected to interact with external environments rather than only generate text. A key challenge in this process is to accurately and efficiently ground high-level agent intents into valid executable action sequences, especially when the environment consists of multiple interacting modules. To address this challenge, we introduce executable action programs (EAPs), a restricted Python-based intermediate representation between textual agent intents and structured environment-module action calls, which leverages the expressive power of Python to explicitly represent complex behavioral sequences, their branching relationships, and cross-module data dependencies. Based on this representation, we implement EAPRouter, which generates, validates, executes, and reuses EAPs through an agent-EAP interface and a lightweight environment module integration protocol. We evaluate EAPRouter on real agent-environment interaction traces from social simulation, a demanding application scenario with multi-agent, multi-module, and repeated interaction patterns. Experiments show that EAPRouter achieves a 60.0% grounding success rate with Kimi-K2-Instruct, outperforming the SOTA function-calling baseline by 31.6%. It also improves over the strongest baselines by 11.5%-25.4% on GLM-4.7, Claude Opus 4.5, and Gemini 2.5 Flash, with smaller gains on less code-capable models. On the cross-domain NESTFUL v2 benchmark, EAPRouter matches ReAct’s executable accuracy (83.8% vs. 84.9%) while reducing LLM calls, tokens, and execution time by about 80%. EAP pre-generation further handles over 70% of interaction requests, cache reuse covers 66.5% of the remaining ones, and safety validation blocks all tested unsafe payloads and prompt-injection attempts. The code is available at https://github.com/tsinghua-fib-lab/EAPRouter.

The development of large language models (LLMs) offers a feasible approach to simulating complex behavioral patterns of individuals, enabling the reconstruction of microscopic and realistic human societal dynamics. However, this approach demands a realistic environment to provide feedback for the evolving of agents, as well as a parallelized framework to support the massive and uncertain interactions among agents and environments. To address the gaps in existing works, which lack real-world environments and struggle with complex interactions, we design a scalable framework named AgentSociety, which integrates realistic societal environments and parallelized interactions to support simulations of large-scale agents. Experiments demonstrate that the framework can support simulations of 30,000 agents that are faster than the wall-clock time with 24 NVIDIA A800 GPUs and the performance grows linearly with the increase of LLM computational resources. We also show that the integration of realistic environments significantly enhances the authenticity of the agents’ behaviors. Through the framework and experimental results, we are confident that deploying large-scale LLM Agents to simulate human societies becomes feasible. This will help practitioners in fields such as social sciences and management sciences to obtain new scientific discoveries via language generation technologies, and even improve planning and decision-making in the real world. The code is available at https://github.com/tsinghua-fib-lab/agentsociety/.