Jiawei Liao

Author directory

2026

Retrieval-based speculative decoding accelerates large language model inference without training an additional draft model, but its effectiveness is often limited by retrieval ambiguity. Existing future-guided methods typically rely on a fixed future-token position, such as a proxy for the next-next token, as the retrieval anchor. However, this fixed-anchor strategy may be suboptimal across different contexts. We propose InfoSpec, a training-free framework that formulates retrieval-anchor selection as an optimization problem performed at inference time. At each decoding step, InfoSpec constructs a lightweight candidate anchor pool from the target model, estimates the retrieval utility of each candidate via hash-index probing, and selects anchors based on model confidence, ambiguity reduction, locality, and continuation consistency under a fixed draft-tree budget. Experiments on generation, reasoning, code, and long-context benchmarks show that InfoSpec improves mean accepted tokens and end-to-end speedup over strong retrieval-based baselines. Further analyses show that InfoSpec reduces high-ambiguity matches and uses the verification budget more effectively while preserving plug-and-play deployment.