<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="fan-etal-2026-beyond">
    <titleInfo>
        <title>Beyond Adapter Expansion: Packing Multiple Tasks into Spare LoRA Capacity</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Shuangjie</namePart>
        <namePart type="family">Fan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chengxiang</namePart>
        <namePart type="family">Huang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Zhiyuan</namePart>
        <namePart type="family">Zhao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Piotr</namePart>
        <namePart type="family">Koniusz</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Junyu</namePart>
        <namePart type="family">Gao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Di</namePart>
        <namePart type="family">Hu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Many continual learning methods for LLMs mitigate forgetting by expanding task-specific modules, yet this leads to memory costs that grow with the number of tasks. We reveal that such expansion may be unnecessary: a single LoRA adapter possesses substantial per-task parameter redundancy, where retaining only 3–10% of LoRA-B entries suffices to recover over 95% of task accuracy. Moreover, the per-parameter update magnitudes follow a heavy-tailed distribution whose geometric knee point naturally separates critical parameters from dispensable ones. Building on this insight, we propose Continual Adaptation via Subspace Trimming (CAST), which adaptively allocates a compact task-specific subspace within a shared LoRA adapter using the knee point criterion, freezes previously assigned parameters to eliminate cross-task interference, and routes inputs to relevant subspaces via learned semantic vectors without requiring task identities. Experiments on LLaMA-3.1-8B and Qwen3-8B demonstrate that CAST matches per-task LoRA accuracy across 15 sequential tasks while maintaining a fixed adapter budget with over 24% idle capacity preserved.</abstract>
    <identifier type="citekey">fan-etal-2026-beyond</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.1570/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>35766</start>
            <end>35783</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Beyond Adapter Expansion: Packing Multiple Tasks into Spare LoRA Capacity
%A Fan, Shuangjie
%A Huang, Chengxiang
%A Zhao, Zhiyuan
%A Koniusz, Piotr
%A Gao, Junyu
%A Hu, Di
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F fan-etal-2026-beyond
%X Many continual learning methods for LLMs mitigate forgetting by expanding task-specific modules, yet this leads to memory costs that grow with the number of tasks. We reveal that such expansion may be unnecessary: a single LoRA adapter possesses substantial per-task parameter redundancy, where retaining only 3–10% of LoRA-B entries suffices to recover over 95% of task accuracy. Moreover, the per-parameter update magnitudes follow a heavy-tailed distribution whose geometric knee point naturally separates critical parameters from dispensable ones. Building on this insight, we propose Continual Adaptation via Subspace Trimming (CAST), which adaptively allocates a compact task-specific subspace within a shared LoRA adapter using the knee point criterion, freezes previously assigned parameters to eliminate cross-task interference, and routes inputs to relevant subspaces via learned semantic vectors without requiring task identities. Experiments on LLaMA-3.1-8B and Qwen3-8B demonstrate that CAST matches per-task LoRA accuracy across 15 sequential tasks while maintaining a fixed adapter budget with over 24% idle capacity preserved.
%U https://aclanthology.org/2026.emnlp-main.1570/
%P 35766-35783

Markdown (Informal)

[Beyond Adapter Expansion: Packing Multiple Tasks into Spare LoRA Capacity](https://aclanthology.org/2026.emnlp-main.1570/) (Fan et al., EMNLP 2026)

• Beyond Adapter Expansion: Packing Multiple Tasks into Spare LoRA Capacity (Fan et al., EMNLP 2026)

ACL

• Shuangjie Fan, Chengxiang Huang, Zhiyuan Zhao, Piotr Koniusz, Junyu Gao, and Di Hu. 2026. Beyond Adapter Expansion: Packing Multiple Tasks into Spare LoRA Capacity. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 35766–35783, Budapest, Hungary. Association for Computational Linguistics.