Large language Models (LLM) routing reduces inference cost by selecting the most cost-effective model per query, but production deployment requires a router that adapts to an evolving model pool while adding negligible per-query overhead. Unsupervised retrieval-based routers satisfy both, but their support corpus must be built online from streaming traffic, and labeling each query requires running every candidate plus an LLM judge: prohibitively expensive at scale. We frame online corpus construction as stream-based active learning, exploiting a structural property of routing benchmarks: one or two models are globally near-optimal on the majority of queries, so the labeling budget should concentrate on the rare exception regions where selecting a specialist matters. We formalize this with a performance sparsity score, defined as the Kullback-Leibler divergence of a query’s per-candidate performance vector from uniform: high when performance concentrates on a few models and low when all candidates are equivalent. Since this score depends on performances unavailable at inference time, we predict it online from query embeddings with a lightweight regressor. Across four benchmarks, our method SparseNN approaches full-annotation routing performance with up to 44% fewer annotations in the stationary setting and 73% under topic shift. The results transfer to a fully held-out benchmark. A partial model annotation variant achieves higher cost savings, especially on large pools. Finally, both the corpus and the online estimator prove robust to model-pool changes.