<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="hedna-etal-2026-self">
    <titleInfo>
        <title>Self-Calibrating Language Models via Test-Time Discriminative Distillation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Mohamed</namePart>
        <namePart type="given">Rissal</namePart>
        <namePart type="family">Hedna</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jan</namePart>
        <namePart type="family">Strich</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Martin</namePart>
        <namePart type="family">Semmann</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chris</namePart>
        <namePart type="family">Biemann</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Large language models (LLMs) are systematically overconfident: they routinely express high certainty on questions they often answer incorrectly. Existing calibration methods either require labeled validation data, degrade under distribution shifts, or incur substantial inference costs. Recent work has shown that LLMs already contain a better-calibrated signal than the one they verbalize: the token probability of “True” when the model is asked “Is this answer correct?” (P(True)) consistently outperforms their stated confidence, a gap that is theoretically grounded as generative error is lower-bounded by roughly twice the corresponding discriminative error. We introduce SECL (SElf-Calibrating Language Models), a test-time training (TTT) pipeline that exploits this gap as self-supervision: adaptation requires no correctness labels or human feedback, with only two hyperparameters set once per model family from a small labeled probe. SECL adapts only when the input distribution shifts, training on just 6–26% of the question stream at lower cost than the baseline it distills from. Across five language models (2B–12B) from four model families and four diverse domains, SECL reduces Expected Calibration Error (ECE) by 56–78%, outperforming its own supervision signal and matching or outperforming recent inference-time methods. SECL is the first method to apply TTT to calibration; seven ablations covering signal quality, gating strategy, weight accumulation, loss design, domain ordering, hyperparameter sensitivity, and layer selection confirm that each component is crucial and robust across configurations. Code: https://github.com/rissalhedna/SECL</abstract>
    <identifier type="citekey">hedna-etal-2026-self</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.96/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>2238</start>
            <end>2266</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Self-Calibrating Language Models via Test-Time Discriminative Distillation
%A Hedna, Mohamed Rissal
%A Strich, Jan
%A Semmann, Martin
%A Biemann, Chris
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F hedna-etal-2026-self
%X Large language models (LLMs) are systematically overconfident: they routinely express high certainty on questions they often answer incorrectly. Existing calibration methods either require labeled validation data, degrade under distribution shifts, or incur substantial inference costs. Recent work has shown that LLMs already contain a better-calibrated signal than the one they verbalize: the token probability of “True” when the model is asked “Is this answer correct?” (P(True)) consistently outperforms their stated confidence, a gap that is theoretically grounded as generative error is lower-bounded by roughly twice the corresponding discriminative error. We introduce SECL (SElf-Calibrating Language Models), a test-time training (TTT) pipeline that exploits this gap as self-supervision: adaptation requires no correctness labels or human feedback, with only two hyperparameters set once per model family from a small labeled probe. SECL adapts only when the input distribution shifts, training on just 6–26% of the question stream at lower cost than the baseline it distills from. Across five language models (2B–12B) from four model families and four diverse domains, SECL reduces Expected Calibration Error (ECE) by 56–78%, outperforming its own supervision signal and matching or outperforming recent inference-time methods. SECL is the first method to apply TTT to calibration; seven ablations covering signal quality, gating strategy, weight accumulation, loss design, domain ordering, hyperparameter sensitivity, and layer selection confirm that each component is crucial and robust across configurations. Code: https://github.com/rissalhedna/SECL
%U https://aclanthology.org/2026.emnlp-main.96/
%P 2238-2266

Markdown (Informal)

[Self-Calibrating Language Models via Test-Time Discriminative Distillation](https://aclanthology.org/2026.emnlp-main.96/) (Hedna et al., EMNLP 2026)

• Self-Calibrating Language Models via Test-Time Discriminative Distillation (Hedna et al., EMNLP 2026)

ACL

• Mohamed Rissal Hedna, Jan Strich, Martin Semmann, and Chris Biemann. 2026. Self-Calibrating Language Models via Test-Time Discriminative Distillation. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 2238–2266, Budapest, Hungary. Association for Computational Linguistics.