<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="durai-2025-phases">
    <titleInfo>
        <title>Phases of Uncertainty: Confidence–Calibration Dynamics in Language Model Training</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Aneesh</namePart>
        <namePart type="family">Durai</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2nd Workshop on Uncertainty-Aware NLP (UncertaiNLP 2025)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Noidea</namePart>
            <namePart type="family">Noidea</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Suzhou, China</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-349-4</identifier>
    </relatedItem>
    <abstract>Autoregressive language models achieve strong performance across a wide range of natural language processing (NLP) tasks, yet their uncertainty estimates remain poorly understood, particularly during training. Prior work has primarily evaluated calibration and out-of-distribution (OOD) robustness at the final checkpoint, overlooking the dynamics that unfold earlier. We introduce a phase-based framework for tracking uncertainty metrics—including expected calibration error (ECE) and Kullback–Leibler (KL) divergence—across distinct stages of training. Using GPT-2 models trained across multiple random seeds, we find that uncertainty dynamics follow a consistent set of phases: models begin conservative and relatively well calibrated, but later phases introduce a paradoxical decoupling where confidence increases even as calibration worsens, especially under distribution shift. This paradox implies that the final checkpoint is not always the most reliable for deployment and motivates phase-aware strategies such as dynamic checkpoint selection or targeted calibration. Our findings highlight that uncertainty should be understood as a training-dependent property rather than a static one, opening new directions for scaling this framework to larger models, tasks, and distribution shift scenarios.</abstract>
    <identifier type="citekey">durai-2025-phases</identifier>
    <location>
        <url>https://aclanthology.org/2025.uncertainlp-main.2/</url>
    </location>
    <part>
        <date>2025-11</date>
        <extent unit="page">
            <start>11</start>
            <end>16</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Phases of Uncertainty: Confidence–Calibration Dynamics in Language Model Training
%A Durai, Aneesh
%Y Noidea, Noidea
%S Proceedings of the 2nd Workshop on Uncertainty-Aware NLP (UncertaiNLP 2025)
%D 2025
%8 November
%I Association for Computational Linguistics
%C Suzhou, China
%@ 979-8-89176-349-4
%F durai-2025-phases
%X Autoregressive language models achieve strong performance across a wide range of natural language processing (NLP) tasks, yet their uncertainty estimates remain poorly understood, particularly during training. Prior work has primarily evaluated calibration and out-of-distribution (OOD) robustness at the final checkpoint, overlooking the dynamics that unfold earlier. We introduce a phase-based framework for tracking uncertainty metrics—including expected calibration error (ECE) and Kullback–Leibler (KL) divergence—across distinct stages of training. Using GPT-2 models trained across multiple random seeds, we find that uncertainty dynamics follow a consistent set of phases: models begin conservative and relatively well calibrated, but later phases introduce a paradoxical decoupling where confidence increases even as calibration worsens, especially under distribution shift. This paradox implies that the final checkpoint is not always the most reliable for deployment and motivates phase-aware strategies such as dynamic checkpoint selection or targeted calibration. Our findings highlight that uncertainty should be understood as a training-dependent property rather than a static one, opening new directions for scaling this framework to larger models, tasks, and distribution shift scenarios.
%U https://aclanthology.org/2025.uncertainlp-main.2/
%P 11-16

Markdown (Informal)

[Phases of Uncertainty: Confidence–Calibration Dynamics in Language Model Training](https://aclanthology.org/2025.uncertainlp-main.2/) (Durai, UncertaiNLP 2025)

• Phases of Uncertainty: Confidence–Calibration Dynamics in Language Model Training (Durai, UncertaiNLP 2025)

ACL

• Aneesh Durai. 2025. Phases of Uncertainty: Confidence–Calibration Dynamics in Language Model Training. In Proceedings of the 2nd Workshop on Uncertainty-Aware NLP (UncertaiNLP 2025), pages 11–16, Suzhou, China. Association for Computational Linguistics.