Maram Assi
Author directory2026
Scaling Enterprise Cloud Agents to Thousands of APIs: An Empirical Study of Agent Architecture and MCP Tool Boundaries
Sandip Ghoshal | Khalil Mrini | Maram Assi | Yadan Fan | Jyotika Singh | Miguel Ballesteros | Weiyi Sun | Anshul Mittal | Yassine Benajiba | Sujeeth Bharadwaj | Sujith Ravi | Dan Roth
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Sandip Ghoshal | Khalil Mrini | Maram Assi | Yadan Fan | Jyotika Singh | Miguel Ballesteros | Weiyi Sun | Anshul Mittal | Yassine Benajiba | Sujeeth Bharadwaj | Sujith Ravi | Dan Roth
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Enterprise cloud agents increasingly operate over thousands of APIs and SDK operations. Tool use therefore becomes a large-scale classification and grounding problem over API operations, not a simple function-calling task. Earlier production-derived architectures either expose broad operation catalogs to the model or hide selection inside planner agents, causing excessive context growth, repeated discovery calls, and preventable invocation errors. We propose the Operation-Visible Agent Architecture, which keeps operation search, schema inspection, validation feedback, and API invocation visible to a centralized ReAct loop through a deterministic Operation-Resolver MCP. We compare this architecture with lead-worker planning and ReAct-based execution variants to assess how agent orchestration and MCP tool-boundary design affect enterprise-agent performance. On internal cloud-management benchmark tasks spanning 280+ SDK clients and approximately 8K operations, the proposed architecture improves full-success rate from 68% to 77% over the production-derived baseline, while reducing total token usage by 54.1%, median latency by 43.3%, LLM calls by 37.8%, and tool errors by 52.4%. The main lesson is that, at enterprise API scale, tool-boundary design is not an implementation detail: parallel tool calling improves efficiency, but task success improves most when operation resolution remains visible to the main reasoning loop.