<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lee-etal-2026-layer">
    <titleInfo>
        <title>Layer-wise Curriculum Learning for Efficient LLM Compression</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Donggeon</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dooyeon</namePart>
        <namePart type="family">Na</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Seungmin</namePart>
        <namePart type="family">Oh</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jongbin</namePart>
        <namePart type="family">Ryu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>In this paper, we introduce layer-wise curriculum learning aimed at efficient LLM compression. The proposed method facilitates the transfer of knowledge from the teacher to student model, utilizing a curriculum learning approach that begins with easier optimization tasks and progressively tackles much harder ones. In order to adopt the layer-wise learning in LLM compression, we partition the whole model into multiple segments consisting of layers, thereby enabling more computationally efficient knowledge transfer of LLMs. Based on the theoretical evidence of cumulative error phenomenon, layer-wise curriculum learning accelerates convergence while stabilizing the knowledge transfer process. In addition, we present a feature caching method with a multi-threading strategy to efficiently address feature misalignment across layers, maximizing GPU utilization. Consequently, our method exhibits advanced model compression performance, as well as high computational efficiency in terms of minimized memory usage and short training hours. Experiments on multiple datasets show that the proposed method achieves state-of-the-art performance while reducing GPU memory usage and training hours by more than 50% on BERT and GPT-2. Moreover, it outperforms the other pruning methods on LLaMA-family and Qwen under the same training hours, with a lower GPU memory footprint. Upon acceptance, we will publicly release the code to support the reproducibility of our results.</abstract>
    <identifier type="citekey">lee-etal-2026-layer</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.2259/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>51372</start>
            <end>51388</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Layer-wise Curriculum Learning for Efficient LLM Compression
%A Lee, Donggeon
%A Na, Dooyeon
%A Oh, Seungmin
%A Ryu, Jongbin
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F lee-etal-2026-layer
%X In this paper, we introduce layer-wise curriculum learning aimed at efficient LLM compression. The proposed method facilitates the transfer of knowledge from the teacher to student model, utilizing a curriculum learning approach that begins with easier optimization tasks and progressively tackles much harder ones. In order to adopt the layer-wise learning in LLM compression, we partition the whole model into multiple segments consisting of layers, thereby enabling more computationally efficient knowledge transfer of LLMs. Based on the theoretical evidence of cumulative error phenomenon, layer-wise curriculum learning accelerates convergence while stabilizing the knowledge transfer process. In addition, we present a feature caching method with a multi-threading strategy to efficiently address feature misalignment across layers, maximizing GPU utilization. Consequently, our method exhibits advanced model compression performance, as well as high computational efficiency in terms of minimized memory usage and short training hours. Experiments on multiple datasets show that the proposed method achieves state-of-the-art performance while reducing GPU memory usage and training hours by more than 50% on BERT and GPT-2. Moreover, it outperforms the other pruning methods on LLaMA-family and Qwen under the same training hours, with a lower GPU memory footprint. Upon acceptance, we will publicly release the code to support the reproducibility of our results.
%U https://aclanthology.org/2026.emnlp-main.2259/
%P 51372-51388

Markdown (Informal)

[Layer-wise Curriculum Learning for Efficient LLM Compression](https://aclanthology.org/2026.emnlp-main.2259/) (Lee et al., EMNLP 2026)

• Layer-wise Curriculum Learning for Efficient LLM Compression (Lee et al., EMNLP 2026)

ACL

• Donggeon Lee, Dooyeon Na, Seungmin Oh, and Jongbin Ryu. 2026. Layer-wise Curriculum Learning for Efficient LLM Compression. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 51372–51388, Budapest, Hungary. Association for Computational Linguistics.