<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="finlay-etal-2026-caliber">
    <titleInfo>
        <title>CALIBER: Calibrating Confidence Before and After Reasoning in Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Conor</namePart>
        <namePart type="family">Finlay</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Joshua</namePart>
        <namePart type="family">Kurien</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Saurabh</namePart>
        <namePart type="family">Dash</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Marzieh</namePart>
        <namePart type="family">Fadaee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Beyza</namePart>
        <namePart type="family">Ermis</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-462-0</identifier>
    </relatedItem>
    <abstract>Reasoning language models are increasingly asked not only to answer difficult questions, but also to estimate their likelihood of success. Existing methods typically elicit confidence only once: either before thinking or after answering. We argue that confidence in reasoning models is state-dependent: before thinking, confidence should estimate whether the current policy will solve the prompt, while after thinking it should predict whether the realized answer is correct. This distinction determines the appropriate supervision target: prompt-level success should supervise confidence estimates made after seeing the prompt, while individual answer-level correctness should supervise confidence estimates made after answering. We introduce CALIBER (Calibration Before and After Reasoning), which elicits both estimates and supervises each with the target matched to its information state. Under this unified protocol, CALIBER reduces Expected Calibration Error (ECE) by 52.5% over the strongest single-confidence baseline on BigMathDigits for a 7B model, while achieving the best Brier score and AUROC, and remains within 2.0 points of the best accuracy. Out of distribution at this scale, it achieves the best ECE and Brier score on all three benchmarks, and ties for the best AUROC on SimpleQA. On a larger 30B model, CALIBER obtains the best ECE and Brier score on all four benchmarks and the best AUROC on three of the four. Ablations further show that this position–target alignment is most beneficial under distribution shift.</abstract>
    <identifier type="citekey">finlay-etal-2026-caliber</identifier>
    <location>
        <url>https://aclanthology.org/2026.findings-emnlp.2420/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>51966</start>
            <end>51980</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T CALIBER: Calibrating Confidence Before and After Reasoning in Language Models
%A Finlay, Conor
%A Kurien, Joshua
%A Dash, Saurabh
%A Fadaee, Marzieh
%A Ermis, Beyza
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Findings of the Association for Computational Linguistics: EMNLP 2026
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-462-0
%F finlay-etal-2026-caliber
%X Reasoning language models are increasingly asked not only to answer difficult questions, but also to estimate their likelihood of success. Existing methods typically elicit confidence only once: either before thinking or after answering. We argue that confidence in reasoning models is state-dependent: before thinking, confidence should estimate whether the current policy will solve the prompt, while after thinking it should predict whether the realized answer is correct. This distinction determines the appropriate supervision target: prompt-level success should supervise confidence estimates made after seeing the prompt, while individual answer-level correctness should supervise confidence estimates made after answering. We introduce CALIBER (Calibration Before and After Reasoning), which elicits both estimates and supervises each with the target matched to its information state. Under this unified protocol, CALIBER reduces Expected Calibration Error (ECE) by 52.5% over the strongest single-confidence baseline on BigMathDigits for a 7B model, while achieving the best Brier score and AUROC, and remains within 2.0 points of the best accuracy. Out of distribution at this scale, it achieves the best ECE and Brier score on all three benchmarks, and ties for the best AUROC on SimpleQA. On a larger 30B model, CALIBER obtains the best ECE and Brier score on all four benchmarks and the best AUROC on three of the four. Ablations further show that this position–target alignment is most beneficial under distribution shift.
%U https://aclanthology.org/2026.findings-emnlp.2420/
%P 51966-51980

Markdown (Informal)

[CALIBER: Calibrating Confidence Before and After Reasoning in Language Models](https://aclanthology.org/2026.findings-emnlp.2420/) (Finlay et al., Findings 2026)

• CALIBER: Calibrating Confidence Before and After Reasoning in Language Models (Finlay et al., Findings 2026)

ACL

• Conor Finlay, Joshua Kurien, Saurabh Dash, Marzieh Fadaee, and Beyza Ermis. 2026. CALIBER: Calibrating Confidence Before and After Reasoning in Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 51966–51980, Budapest, Hungary. Association for Computational Linguistics.