Avery Nie
Author directory2026
AsyncTool: Evaluating the Asynchronous Function Calling Capability under Multi-Task Scenarios
Kou Shi | Ziao Zhang | Shiting Huang | Avery Nie | Zhen Fang | Qiuchen Wang | Lin Chen | Huaian Chen | Zehui Chen | Feng Zhao
Findings of the Association for Computational Linguistics: EMNLP 2026
Kou Shi | Ziao Zhang | Shiting Huang | Avery Nie | Zhen Fang | Qiuchen Wang | Lin Chen | Huaian Chen | Zehui Chen | Feng Zhao
Findings of the Association for Computational Linguistics: EMNLP 2026
Large language model (LLM)-based agents have demonstrated strong capabilities in leveraging external tools to solve complex tasks. However, existing evaluations largely overlook the temporal dimension of tool invocation, particularly the impact of tool response latency, and are typically limited to single-task settings. In real-world applications, multiple tasks often need to be executed concurrently, and overall efficiency critically depends on whether an agent can utilize idle time while waiting for tool responses. We refer to this capability as asynchronous tool calling. To evaluate this capability, we propose AsyncTool, a benchmark for assessing LLM-based agents in interactive multi-task tool-use environments with delayed tool feedback. AsyncTool presents multiple heterogeneous tasks simultaneously and simulates realistic tool response latency during execution. Using a hybrid data evolution strategy, we construct a diverse asynchronous multitasking dataset covering multiple scenarios and tool-use patterns. We evaluate models at three levels—step, sub-task, and task—and further introduce efficiency-oriented metrics to measure task coordination and completion efficiency. Extensive experiments show that delayed tool feedback poses substantial challenges to current agents, leading to clear performance degradation. Models that better coordinate task switching and dependency tracking tend to achieve stronger performance on AsyncTool.Our analysis identifies the main failure modes of current tool agents and provides practical guidelines for designing future systems with stronger temporal reasoning and coordination capabilities.