Other people with similar names: Han Luo


2026

Cost-effective deployment of large language models (LLMs) often relies on routing each query to either a low-cost local model or a stronger expert model. Existing routers typically decide either before generation, which is cheap but cannot observe the local model’s answer, or after generation, which provides a stronger correctness signal but incurs additional generation cost. We argue that this dichotomy overlooks a third option: treating local generation itself as an optional information-acquisition action. To this end, we propose Generation-Gated Cascaded Routing for Large Language Models (GenGate), a lightweight fine-tuning method that trains a local LLM to internalize cascaded routing decisions while preserving its generation capability. A Bellman-motivated prior gate first decides whether to generate a local attempt. If an attempt is produced, a correctness-supervised posterior gate then decides whether to submit it or defer to the expert model. Both decisions are expressed through action-token probabilities in the same model, requiring no external router or auxiliary classifier at inference time. Compared with routers that decide strictly before or after local generation, GenGate reduces joint deployment cost while maintaining competitive routing accuracy across task domains, preserving the benefits of posterior routing without requiring local generation for every query.