<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="luong-chen-2026-rethinking">
    <titleInfo>
        <title>Rethinking the Role of Temperature in Large Language Model Distillation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hoang-Chau</namePart>
        <namePart type="family">Luong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lingwei</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Junyi</namePart>
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Reverse Kullback–Leibler (RKL) divergence is widely favored over forward KL (FKL) in large language models (LLM) distillation, yet this preference is largely based on comparisons that omit the temperature τ, overlooking its central role in softening teacher distributions and improving knowledge transfer. In this work, we revisit temperature in LLM distillation and show that it fundamentally changes the comparison between FKL and RKL. Our analysis reveals an asymmetric effect: temperature substantially enriches FKL with non-dominant token signals, whereas it mainly rescales RKL gradients, causing FKL to benefit much more from τ scaling than RKL. This asymmetry overturns the standard empirical conclusion: although RKL outperforms FKL at τ=1, FKL consistently surpasses RKL at higher temperatures across instruction-following benchmarks. Moreover, the impact of temperature is not limited to FKL; it improves a broader family of distillation objectives, enabling simple KL-based methods to achieve competitive performance against recent state-of-the-art LLM distillation approaches.</abstract>
    <identifier type="citekey">luong-chen-2026-rethinking</identifier>
    <identifier type="doi">10.18653/2026.emnlp-main.1191</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.1191/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>27292</start>
            <end>27303</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Rethinking the Role of Temperature in Large Language Model Distillation
%A Luong, Hoang-Chau
%A Chen, Lingwei
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Junyi Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F luong-chen-2026-rethinking
%X Reverse Kullback–Leibler (RKL) divergence is widely favored over forward KL (FKL) in large language models (LLM) distillation, yet this preference is largely based on comparisons that omit the temperature τ, overlooking its central role in softening teacher distributions and improving knowledge transfer. In this work, we revisit temperature in LLM distillation and show that it fundamentally changes the comparison between FKL and RKL. Our analysis reveals an asymmetric effect: temperature substantially enriches FKL with non-dominant token signals, whereas it mainly rescales RKL gradients, causing FKL to benefit much more from τ scaling than RKL. This asymmetry overturns the standard empirical conclusion: although RKL outperforms FKL at τ=1, FKL consistently surpasses RKL at higher temperatures across instruction-following benchmarks. Moreover, the impact of temperature is not limited to FKL; it improves a broader family of distillation objectives, enabling simple KL-based methods to achieve competitive performance against recent state-of-the-art LLM distillation approaches.
%R 10.18653/2026.emnlp-main.1191
%U https://aclanthology.org/2026.emnlp-main.1191/
%U https://doi.org/10.18653/2026.emnlp-main.1191
%P 27292-27303

Markdown (Informal)

[Rethinking the Role of Temperature in Large Language Model Distillation](https://aclanthology.org/2026.emnlp-main.1191/) (Luong & Chen, EMNLP 2026)

• Rethinking the Role of Temperature in Large Language Model Distillation (Luong & Chen, EMNLP 2026)

ACL

• Hoang-Chau Luong and Lingwei Chen. 2026. Rethinking the Role of Temperature in Large Language Model Distillation. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 27292–27303, Budapest, Hungary. Association for Computational Linguistics.