Large Language Models (LLMs) are strong at multi-step reasoning, but running them for every generated token is expensive. Small language models (SLMs) are much cheaper, yet less reliable at the hardest reasoning steps. Existing routing methods mostly make a single query-level decision, assuming that response difficulty is uniform once decoding begins. For chain-of-thought reasoning, however, difficulty varies during generation, making token-level routing a more appropriate granularity. This setting introduces three challenges: token-level uncertainty is noisy, local routing decisions must satisfy global efficiency constraints, and frequent switching can disrupt reasoning coherence. We present TokenRouter, a black-box runtime controller for uncertainty-guided token-level routing between small and large language models. TokenRouter combines a robust entropy detector, a self-calibrating control policy, and a dynamic intervention commitment mechanism for stable switching over difficult spans. Across three reasoning benchmarks, TokenRouter maintains over 99% of pure-LLM accuracy while reducing large-model activation to 56.2%–76.0% of generated tokens. Code is available at https://github.com/sznnzs/TokenRouter.

KV cache compression is critical for long-context LLM inference, yet existing training-free methods fall into a dilemma: attention-based eviction clusters retained tokens around semantically “hot” regions, creating context holes that degrade long-range reasoning, while diversity-based selection retains directionally broad but semantically irrelevant tokens. We identify the root cause: at prefill time, future decoding queries are unknown, making query-conditioned signals unreliable as a sole selection criterion. The retained KV set must instead provide broad directional coverage of the context—so that arbitrary future queries can find supporting keys—while filtering out tokens that expand the coverage span without contributing to generation. We propose AnchorKV, which realizes this principle through block-wise partitioning with uniform per-block budgets and anchor-driven cosine dissimilarity scoring for linear-time diversity estimation. A compression-rate adaptive hybrid score dynamically reweights diversity and importance. Experiments on LongBench across three model families show AnchorKV consistently outperforms existing training-free methods, with the largest margins at low retention ratios. A query-subsampled variant reduces prefill latency by up to 2.7× with minimal accuracy loss. We provide theoretical guarantees on approximation error and variance reduction.

Large language models (LLMs) excel at complex reasoning tasks but often suffer from overconfidence and computational inefficiency due to fixed computation budgets and miscalibrated confidence estimates. We present a novel framework for computationally efficient, trustworthy reasoning under uncertainty, introducing two complementary techniques: Diversity-Aware Self-Signal Dilution (DASD) and Convergent Adaptive Weighted Sampling (CAWS). DASD operates in an unsupervised manner to dilute overconfident, semantically redundant reasoning paths, thereby producing better-calibrated internal confidence estimates. CAWS dynamically allocates computational resources at inference time by aggregating these signals and terminating computation once answer dominance and stability are achieved. Comprehensive experiments across three reasoning datasets demonstrate that our approach maintains accuracy levels while achieving over 70% reduction in inference cost, surpassing competitive baselines. Our framework provides a scalable, unsupervised solution for reliable and efficient LLM reasoning.