Jinru Han
Author directory2026
Routing-Free Mixture of Experts
Yilun Liu | Jinru Han | Sikuan Yan | Volker Tresp | Yunpu Ma
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yilun Liu | Jinru Han | Sikuan Yan | Volker Tresp | Yunpu Ma
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Standard Mixture-of-Experts (MoE) designs rely heavily on centralized routing mechanisms that introduce rigid inductive biases. We propose Routing-Free MoE, which eliminates all hard-coded centralized components, including external routers, Softmax, TopK, and fixed load balancing. All related functionalities become encapsulated within individual experts, allowing each expert to determine its activation entirely on its own. We introduce a unified, adaptively scheduled load-balancing framework to simultaneously optimize both expert-balancing and token-balancing objectives through a customizable interpolation, letting optimal resource allocation pattern emerge locally during training. Extensive experiments demonstrate that Routing-Free MoE achieves consistently better performance, efficiency, and robustness than baseline approaches at both training and deployment. We further analyze its emergent behavior in detail and offer insights that may facilitate future MoE design and optimization. Code is available at https://liuyilun2000.github.io/RoutingFreeMoE/.