<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="li-etal-2026-calibrated">
    <titleInfo>
        <title>Calibrated Tree-Structured Speculative Decoding: Mitigating Overconfidence via Path-Level Calibration</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Zixian</namePart>
        <namePart type="family">LI</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Xiaohui</namePart>
        <namePart type="family">Song</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chi</namePart>
        <namePart type="family">Xie</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ke</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chen</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Haonan</namePart>
        <namePart type="family">Lu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Tree-structured speculative decoding accelerates large language model inference by drafting multiple candidate tokens in parallel and verifying them against a target model. Dynamic tree expansion strategies further improve efficiency by using draft model confidence to prune unlikely branches and stop expansion early. However, we find that existing draft models exhibit severe overconfidence: they assign high probabilities to tokens that will be rejected, causing unreliable early stopping and wasted computation on incorrect branches.\par We propose Calibrated Tree-Structured Speculative Decoding (CATS-SD), a training framework for tree-based confidence calibration. Our approach calibrates draft model confidence at two complementary levels: (1) token-level calibration that aligns draft-target token ranking, and (2) path-level calibration that supervises confidence over tree-structured decoding paths. We also introduce a calibration metric using Pearson correlation to quantify confidence-acceptance alignment. On Qwen2.5-3B-Instruct, CATS-SD achieves up to 11.3% higher acceptance length and 4.0% better confidence-acceptance correlation compared to the baseline, with end-to-end speedup improving from 2.51\times to 2.71\times on GSM8K. We validate these results across mathematical reasoning (GSM8K), multi-turn conversation (MT-bench), and code generation (HumanEval).</abstract>
    <identifier type="citekey">li-etal-2026-calibrated</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.80/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>1844</start>
            <end>1855</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Calibrated Tree-Structured Speculative Decoding: Mitigating Overconfidence via Path-Level Calibration
%A LI, Zixian
%A Song, Xiaohui
%A Xie, Chi
%A Chen, Ke
%A Chen, Chen
%A Lu, Haonan
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F li-etal-2026-calibrated
%X Tree-structured speculative decoding accelerates large language model inference by drafting multiple candidate tokens in parallel and verifying them against a target model. Dynamic tree expansion strategies further improve efficiency by using draft model confidence to prune unlikely branches and stop expansion early. However, we find that existing draft models exhibit severe overconfidence: they assign high probabilities to tokens that will be rejected, causing unreliable early stopping and wasted computation on incorrect branches.\par We propose Calibrated Tree-Structured Speculative Decoding (CATS-SD), a training framework for tree-based confidence calibration. Our approach calibrates draft model confidence at two complementary levels: (1) token-level calibration that aligns draft-target token ranking, and (2) path-level calibration that supervises confidence over tree-structured decoding paths. We also introduce a calibration metric using Pearson correlation to quantify confidence-acceptance alignment. On Qwen2.5-3B-Instruct, CATS-SD achieves up to 11.3% higher acceptance length and 4.0% better confidence-acceptance correlation compared to the baseline, with end-to-end speedup improving from 2.51\times to 2.71\times on GSM8K. We validate these results across mathematical reasoning (GSM8K), multi-turn conversation (MT-bench), and code generation (HumanEval).
%U https://aclanthology.org/2026.emnlp-main.80/
%P 1844-1855

Markdown (Informal)

[Calibrated Tree-Structured Speculative Decoding: Mitigating Overconfidence via Path-Level Calibration](https://aclanthology.org/2026.emnlp-main.80/) (LI et al., EMNLP 2026)

• Calibrated Tree-Structured Speculative Decoding: Mitigating Overconfidence via Path-Level Calibration (LI et al., EMNLP 2026)

ACL

• Zixian LI, Xiaohui Song, Chi Xie, Ke Chen, Chen Chen, and Haonan Lu. 2026. Calibrated Tree-Structured Speculative Decoding: Mitigating Overconfidence via Path-Level Calibration. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 1844–1855, Budapest, Hungary. Association for Computational Linguistics.