<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="li-etal-2026-complexity">
    <titleInfo>
        <title>Complexity-Aware Routing: Detecting and Allocating Dynamic Compute for Efficient Mixture-of-Experts Large Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Jiahe</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Saisai</namePart>
        <namePart type="family">Hu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Menghao</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Shaolin</namePart>
        <namePart type="family">Zhu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Deyi</namePart>
        <namePart type="family">Xiong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Current Mixture-of-Experts (MoE) models typically rely on a static Top-k routing mechanism, activating a fixed number of experts for every token regardless of its difficulty. This uniform allocation ignores the inherent variability in linguistic information density that simple functional words require significantly less processing capacity than content-heavy terms. The static routing results in computational waste for easy tokens and potential under-fitting for difficult ones. To address this issue, we propose **CAR**, a Complexity-Aware Routing framework that aligns the computational budget with the intrinsic information density of input tokens. Diverging from previous dynamic methods that rely on unstable heuristics, we ground our routing decision in a dual-signal objective combining Shannon Surprisal and Predictive Entropy to objectively measure token complexity. We implement a decoupled architecture where a Complexity Perception (CP) module utilizes a Recurrent Complexity Memory (RCM) to estimate these signals from a teacher model, guiding a Dynamic Expert Allocator (DEA) to adjust k on the fly. Extensive experiments demonstrate that CAR consistently outperforms both static baselines and SOTA dynamic routing methods. Notably, our approach achieves superior accuracy on reasoning-intensive benchmarks (e.g., boosting MMLU by 0.8 and GSM8K by 1.5 over static routing) while reducing the average number of activated parameters by approximately 10%. Our code is available at https://github.com/Acking646/car.</abstract>
    <identifier type="citekey">li-etal-2026-complexity</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.1206/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>27594</start>
            <end>27610</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Complexity-Aware Routing: Detecting and Allocating Dynamic Compute for Efficient Mixture-of-Experts Large Language Models
%A Li, Jiahe
%A Hu, Saisai
%A Li, Menghao
%A Zhu, Shaolin
%A Xiong, Deyi
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F li-etal-2026-complexity
%X Current Mixture-of-Experts (MoE) models typically rely on a static Top-k routing mechanism, activating a fixed number of experts for every token regardless of its difficulty. This uniform allocation ignores the inherent variability in linguistic information density that simple functional words require significantly less processing capacity than content-heavy terms. The static routing results in computational waste for easy tokens and potential under-fitting for difficult ones. To address this issue, we propose **CAR**, a Complexity-Aware Routing framework that aligns the computational budget with the intrinsic information density of input tokens. Diverging from previous dynamic methods that rely on unstable heuristics, we ground our routing decision in a dual-signal objective combining Shannon Surprisal and Predictive Entropy to objectively measure token complexity. We implement a decoupled architecture where a Complexity Perception (CP) module utilizes a Recurrent Complexity Memory (RCM) to estimate these signals from a teacher model, guiding a Dynamic Expert Allocator (DEA) to adjust k on the fly. Extensive experiments demonstrate that CAR consistently outperforms both static baselines and SOTA dynamic routing methods. Notably, our approach achieves superior accuracy on reasoning-intensive benchmarks (e.g., boosting MMLU by 0.8 and GSM8K by 1.5 over static routing) while reducing the average number of activated parameters by approximately 10%. Our code is available at https://github.com/Acking646/car.
%U https://aclanthology.org/2026.emnlp-main.1206/
%P 27594-27610

Markdown (Informal)

[Complexity-Aware Routing: Detecting and Allocating Dynamic Compute for Efficient Mixture-of-Experts Large Language Models](https://aclanthology.org/2026.emnlp-main.1206/) (Li et al., EMNLP 2026)

• Complexity-Aware Routing: Detecting and Allocating Dynamic Compute for Efficient Mixture-of-Experts Large Language Models (Li et al., EMNLP 2026)

ACL

• Jiahe Li, Saisai Hu, Menghao Li, Shaolin Zhu, and Deyi Xiong. 2026. Complexity-Aware Routing: Detecting and Allocating Dynamic Compute for Efficient Mixture-of-Experts Large Language Models. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 27594–27610, Budapest, Hungary. Association for Computational Linguistics.