Ashwin Chandramouli

Author directory

2026

Fine-tuned small language models (SLMs) are attractive defaults for large-scale structured at- tribute generation—producing values such as materials, colors, or dimensions from product descriptions—as they are inexpensive to serve. However, a fine-tuned SLM is not reliable ev- erywhere as its correctness varies sharply, so deploying it still requires deciding, per output, whether to keep the SLM’s value or escalate it to a more capable fallback. The standard pol- icy applies a single confidence threshold, but one threshold cannot separate scores on reli- able tasks from those on unreliable ones, so it reduces coverage at no gain in quality. We demonstrate that in a fixed deployment scope, where the target tasks recur and validation data is available, a lightweight scope-aware router using only SLM-side features (token confi- dence plus task-scope identity) selects risky predictions far better. Our experiments show that at a fixed precision target it accepts about 11 points more values than a global confidence threshold (family-level calibration helps only marginally). Ablation experiments confirm the gain comes from task-scope reliability priors rather than a better confidence transformation. The router is an 8-feature gradient-boosted tree that adds negligible inference cost, making a strong fine-tuned SLM the low-cost tier of a cascade that optimizes the quality–cost tradeoff in real production settings.