2026

Output-token budgets are a first-order deployment constraint for large language model (LLM) agents, yet existing benchmarks typically evaluate them under a single fixed or generous budget. We introduce TokenCliff, a benchmark for measuring multi-turn agent performance across episode-level output-token budgets. It contains 660 deterministic tasks spanning four capability-oriented domains—file operations, data transformation, tool use, and planning—plus a compositional multi-step split. We propose two budget-sensitivity metrics: Budget Elasticity, which captures robustness across budget levels, and the Cliff Index, which captures the sharpest adjacent-budget performance drop. Evaluating 10 LLMs at five budget levels from 200 to 8,192 tokens yields 33,000 episodes. Results show that (1) accuracy drops by up to 32 percentage points between adjacent budgets; (2) neither peak performance nor larger model size necessarily implies stronger budget elasticity; and (3) budget sensitivity is substantially greater in the evaluated multi-turn setting than in the single-shot diagnostic. Code and data are available in our public repository: https://github.com/BairuiZhang/TokenCliff