Also published as: Sicong Xie
2026
Causal Abstention for Cost-Aware Language-Agent Collaboration
Lei Wei | Ruiyang Huang | Zhaolu Kang | Qi Liu | Xiao Peng | Haoran Liu | Zonghao Ying | Haoyu Wang | Ye Shen | Xu Dong | Hongyu Li | Kesu Wang | Yukun Lian | Sicong Xie
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Lei Wei | Ruiyang Huang | Zhaolu Kang | Qi Liu | Xiao Peng | Haoran Liu | Zonghao Ying | Haoyu Wang | Ye Shen | Xu Dong | Hongyu Li | Kesu Wang | Yukun Lian | Sicong Xie
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Multi-agent language systems are often deployed under the assumption that collaboration improves reasoning quality, but this assumption breaks down once token cost, latency, and collaboration-induced errors enter the picture. We study protocol selection under a cost-aware utility objective, where the system decides on each input whether to run a collaborative protocol or fall back to solo reasoning. We propose Causal Advantage Routing for Language agents (CARL), a contextual-bandit framework that estimates the conditional collaboration advantage of each protocol over solo reasoning. CARL combines confounded execution logs with budgeted randomized probes, forms one-vs-solo doubly robust scores and shrunken DR-derived pseudo-outcomes, and at deployment uses a one-sided predictive lower bound for a future realized protocol-solo contrast, otherwise abstaining to solo execution. Across math, code, and multi-hop question-answering benchmarks with an open-weight model, CARL improves cost-aware utility and substantially reduces harmful collaboration over fixed protocols, compute-matched solo strategies, MAS routers, cascade methods, and causal routing baselines, indicating that effective multi-agent systems should estimate when collaboration is causally worth its cost rather than invoking it by default.
PACE: Predictive Adaptive Context Extraction for Long-Horizon LLM Agents
Lei Wei | Xiao Peng | Tt | Guannan Zhang | Chenhao Jiang | Hongyu Li | Lanbo Lin | Yuanwu Xu | Jiayao Liu | Kesu Wang | Bin Wang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Lei Wei | Xiao Peng | Tt | Guannan Zhang | Chenhao Jiang | Hongyu Li | Lanbo Lin | Yuanwu Xu | Jiayao Liu | Kesu Wang | Bin Wang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Large Language Model (LLM) agents struggle with ultra-long-horizon tasks requiring hundreds or thousands of interaction steps. Traditional context management approaches face a fundamental dilemma: preserving complete histories rapidly exhausts context windows and forces crude truncation, while aggressive summarization discards critical information prematurely. We propose Predictive Adaptive Context Extraction (PACE), a novel framework that reconceptualizes context management as a Next Step Prediction problem. Inspired by neural attention, PACE dynamically constructs context by adjusting historical memory granularity based on its predicted relevance for the next action. Comprehensive evaluation across diverse benchmarks and models demonstrates that PACE consistently improves task success rates, with larger gains on complex tasks and robust cross-lingual performance. Crucially, PACE enables agents to sustain effective reasoning for 4,897 interaction steps in ultra-long-horizon scenarios, achieving a 66.2 improvement over the full-context ReAct baseline and 5.1 over advanced folding baselines. This fundamentally advances the capability of LLM-based agents in previously intractable long-horizon scenarios. Our code and data are available at https://anonymous.4open.science/r/PACE-B000/.