<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="ryu-lee-2026-distribution">
    <titleInfo>
        <title>Out-of-Distribution Checkpoint Selection Has a Resolution Problem: Auditing Sparse Exact Match with Token Likelihood</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Kunhee</namePart>
        <namePart type="family">Ryu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Keeheon</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-462-0</identifier>
    </relatedItem>
    <abstract>Exact match (EM) can make checkpoint selection ambiguous when a small labeled out of distribution (OOD) validation set leaves several checkpoints tied at zero or at the maximum. Without an explicit tie rule, the reported metric does not identify which checkpoint produced the final test score. We study this validation resolution problem in semantic parsing and morphological reinflection. Our protocol reports the reference budget, token likelihood choice under teacher forcing, and latest tied candidate, while using EM for a unique nonzero maximum. Across 600 draws from 30 runs, 8 references leave every early checkpoint at zero in 48% of draws and the maximum tied in 57%. Within tied draws, mean full split regret falls from 0.063 under random selection to 0.050 with token likelihood. The latest tied candidate performs better on average for mostly monotone trajectories, but token likelihood independently orders candidates that hard EM cannot distinguish. These results support reporting both orderings when small labeled OOD validation sets guide checkpoint selection.</abstract>
    <identifier type="citekey">ryu-lee-2026-distribution</identifier>
    <location>
        <url>https://aclanthology.org/2026.findings-emnlp.878/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>18535</start>
            <end>18552</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Out-of-Distribution Checkpoint Selection Has a Resolution Problem: Auditing Sparse Exact Match with Token Likelihood
%A Ryu, Kunhee
%A Lee, Keeheon
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Findings of the Association for Computational Linguistics: EMNLP 2026
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-462-0
%F ryu-lee-2026-distribution
%X Exact match (EM) can make checkpoint selection ambiguous when a small labeled out of distribution (OOD) validation set leaves several checkpoints tied at zero or at the maximum. Without an explicit tie rule, the reported metric does not identify which checkpoint produced the final test score. We study this validation resolution problem in semantic parsing and morphological reinflection. Our protocol reports the reference budget, token likelihood choice under teacher forcing, and latest tied candidate, while using EM for a unique nonzero maximum. Across 600 draws from 30 runs, 8 references leave every early checkpoint at zero in 48% of draws and the maximum tied in 57%. Within tied draws, mean full split regret falls from 0.063 under random selection to 0.050 with token likelihood. The latest tied candidate performs better on average for mostly monotone trajectories, but token likelihood independently orders candidates that hard EM cannot distinguish. These results support reporting both orderings when small labeled OOD validation sets guide checkpoint selection.
%U https://aclanthology.org/2026.findings-emnlp.878/
%P 18535-18552

Markdown (Informal)

[Out-of-Distribution Checkpoint Selection Has a Resolution Problem: Auditing Sparse Exact Match with Token Likelihood](https://aclanthology.org/2026.findings-emnlp.878/) (Ryu & Lee, Findings 2026)

• Out-of-Distribution Checkpoint Selection Has a Resolution Problem: Auditing Sparse Exact Match with Token Likelihood (Ryu & Lee, Findings 2026)

ACL

• Kunhee Ryu and Keeheon Lee. 2026. Out-of-Distribution Checkpoint Selection Has a Resolution Problem: Auditing Sparse Exact Match with Token Likelihood. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 18535–18552, Budapest, Hungary. Association for Computational Linguistics.