The generation quality of large language models (LLMs) is often improved by utilizing inference-time scaling methods such as Chain-of-Thought (sequential scaling) or Self-Consistency (parallel scaling). We introduce Roster of Experts (RoE), a training-free inference algorithm for Mixture-of-Experts (MoE) models that defines a third, orthogonal scaling direction. Rather than generating longer outputs or aggregating multiple sequences, RoE improves the next token prediction distribution by treating a single MoE as a dynamic ensemble of internal subnetworks. It injects controlled stochasticity into the expert routing mechanism via Gumbel-Top-k sampling, enabling the model to aggregate multiple diverse subnetworks into a single, higher-quality token prediction. Crucially, because RoE improves the model’s output distribution rather than sampling from it, it is the only approach in this family that is applicable to open-ended generation tasks where sequence-level aggregation is infeasible. Furthermore, RoE is orthogonal to existing scaling methods; combining RoE with Self-Consistency yields greater accuracy than either method alone. To overcome the computational cost of multiple forward passes at inference, we employ an efficient batching strategy that exploits hardware parallelism to minimize latency, requiring no fine-tuning of model parameters.