LLM agents are increasingly expected to support enterprise workflows, where tasks often involve missing information, uncertainty, feedback, and long-term trade-offs. However, existing enterprise and financial benchmarks mainly test static capabilities such as information extraction, numerical calculation, domain knowledge, and financial QA, leaving interactive and long-horizon decision-making underexplored. To bridge this gap, we introduce EnterpriseBench, a benchmark that evaluates LLM agents across this spectrum, from static question answering to dynamic decision-making. Specifically, EnterpriseBench reorganizes existing enterprise and financial QA datasets into a unified foundational suite annotated by capability and difficulty, and introduces three professional interactive settings: Consulting, based on management-consulting-style business cases for client problem diagnosis through multi-turn information seeking; the Beer Game, adapted from a classic supply-chain management simulation for inventory control under delayed feedback; and Enterprise Digital Twin, a project-based business simulator for workforce, risk, and project planning. Experiments with nine agent methods under two backbone models show that current agents have not yet achieved stable, comprehensive, and cross-task reliability in enterprise scenarios. These results show that EnterpriseBench provides a practical benchmark for evaluating LLM agents in realistic enterprise strategic reasoning and decision-making. Our codes and benchmark implementation are publicly available at https://github.com/sduyangmin/FirmBench.

Large Language Model (LLM) agents are increasingly expected to interact with external environments rather than only generate text. A key challenge in this process is to accurately and efficiently ground high-level agent intents into valid executable action sequences, especially when the environment consists of multiple interacting modules. To address this challenge, we introduce executable action programs (EAPs), a restricted Python-based intermediate representation between textual agent intents and structured environment-module action calls, which leverages the expressive power of Python to explicitly represent complex behavioral sequences, their branching relationships, and cross-module data dependencies. Based on this representation, we implement EAPRouter, which generates, validates, executes, and reuses EAPs through an agent-EAP interface and a lightweight environment module integration protocol. We evaluate EAPRouter on real agent-environment interaction traces from social simulation, a demanding application scenario with multi-agent, multi-module, and repeated interaction patterns. Experiments show that EAPRouter achieves a 60.0% grounding success rate with Kimi-K2-Instruct, outperforming the SOTA function-calling baseline by 31.6%. It also improves over the strongest baselines by 11.5%-25.4% on GLM-4.7, Claude Opus 4.5, and Gemini 2.5 Flash, with smaller gains on less code-capable models. On the cross-domain NESTFUL v2 benchmark, EAPRouter matches ReAct’s executable accuracy (83.8% vs. 84.9%) while reducing LLM calls, tokens, and execution time by about 80%. EAP pre-generation further handles over 70% of interaction requests, cache reuse covers 66.5% of the remaining ones, and safety validation blocks all tested unsafe payloads and prompt-injection attempts. The code is available at https://github.com/tsinghua-fib-lab/EAPRouter.

Large language models (LLMs) are increasingly used for program-aided reasoning, agentic decision making, and structured task execution, but these settings often incur substantial inference cost. Many such requests share similar computational structures while differing in variables, constraints, or contexts, creating opportunities for program-level caching. Since program caches need to reapply reusable computation logic to new requests, their key steps often involve lightweight and structured operations such as variable extraction, program binding, and generation acceleration, which are well suited for small models. We propose CacheSpec, an inference optimization framework centered on reusable program caches. The framework converts Program-of-Thoughts (PoT)-style programs from one-time reasoning artifacts into reusable cache objects, and reuses the same small model for two roles: semantic variable extraction on the cache-hit path and speculative drafting during target-LLM generation. Experiments on shopping-style request datasets, WebShop, Formula, and CodeTATQA show that CacheSpec reduces inference latency and improves effective cache reuse while preserving comparable or better task quality than existing caching and generation baselines, achieving up to about 3.1× latency speedup; in parallel serving experiments, it improves throughput by about 2.8× over PoT-style methods. These results suggest that the sweet spot for small models in large-model inference systems lies not in solving complex tasks independently, but in performing lightweight, structured, and verifiable auxiliary operations.

The development of large language models (LLMs) offers a feasible approach to simulating complex behavioral patterns of individuals, enabling the reconstruction of microscopic and realistic human societal dynamics. However, this approach demands a realistic environment to provide feedback for the evolving of agents, as well as a parallelized framework to support the massive and uncertain interactions among agents and environments. To address the gaps in existing works, which lack real-world environments and struggle with complex interactions, we design a scalable framework named AgentSociety, which integrates realistic societal environments and parallelized interactions to support simulations of large-scale agents. Experiments demonstrate that the framework can support simulations of 30,000 agents that are faster than the wall-clock time with 24 NVIDIA A800 GPUs and the performance grows linearly with the increase of LLM computational resources. We also show that the integration of realistic environments significantly enhances the authenticity of the agents’ behaviors. Through the framework and experimental results, we are confident that deploying large-scale LLM Agents to simulate human societies becomes feasible. This will help practitioners in fields such as social sciences and management sciences to obtain new scientific discoveries via language generation technologies, and even improve planning and decision-making in the real world. The code is available at https://github.com/tsinghua-fib-lab/agentsociety/.