AdaSpec: Adaptive Routing by Ranking for Speculative Agents

Kangqi Ni, Wenyue Hua, Jinbiao Wei, Tianlong Chen


Abstract
Long-horizon LLM agents execute sequences of reasoning, tool-use, and environment-interaction steps, making end-to-end latency a central deployment bottleneck. Speculative agents reduce this latency by leveraging a faster draft agent to propose the action trajectory while a stronger target agent verifies each action to ensure lossless action trajectories. Recent systems adapt how many future actions a preselected draft agent proposes before verification, but leave the drafter itself unchanged across tasks and episodes. Our experiments show that the best drafter varies across tasks and trajectory prefixes, revealing substantial latency headroom from adaptive drafter selection. This motivates the question: which draft agent should speculate for a given task and target agent to achieve low end-to-end latency? We propose AdaSpec, an adaptive routing framework for speculative agents with an objective distinct from standard model routing: selecting the drafter that minimizes realized speculative-execution latency for a target agent, rather than selecting the model that produces the final answer. AdaSpec profiles a pool of candidate draft-agent configurations for each target agent and trains a lightweight, latency-supervised router to rank them by measured speculative latency. The router supports both query-level routing and episode-level rerouting before each draft-and-check segment, while the target agent remains responsible for verifying actions. AdaSpec improves fixed-drafter baselines across multiple datasets and target-agent settings.
Anthology ID:
2026.findings-emnlp.860
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
18196–18211
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.860/
DOI:
Bibkey:
Cite (ACL):
Kangqi Ni, Wenyue Hua, Jinbiao Wei, and Tianlong Chen. 2026. AdaSpec: Adaptive Routing by Ranking for Speculative Agents. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 18196–18211, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
AdaSpec: Adaptive Routing by Ranking for Speculative Agents (Ni et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.860.pdf