<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kubo-etal-2026-rethinking">
    <titleInfo>
        <title>Rethinking Binary Evaluation of Turn-Taking under Inherent Ambiguity</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yunosuke</namePart>
        <namePart type="family">Kubo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kenta</namePart>
        <namePart type="family">Yamamoto</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ryu</namePart>
        <namePart type="family">Takeda</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kazunori</namePart>
        <namePart type="family">Komatani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Jinho</namePart>
            <namePart type="given">D</namePart>
            <namePart type="family">Choi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yun-Nung</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kotaro</namePart>
            <namePart type="family">Funakoshi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ali</namePart>
            <namePart type="family">Emami</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Atlanta, Georgia, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Turn-taking prediction models output probabilities of turn shifts, yet they are typically evaluated by thresholding these probabilities into binary decisions and comparing them against corpus-observed labels. This practice implicitly treats corpus-observed turn shifts as definitive ground truth, even though under inherent turn-taking ambiguity they reflect one realized interactional outcome among multiple plausible outcomes, rather than a uniquely correct binary label. We argue that binary evaluation is a practical simplification rather than a theoretical necessity. Instead, predicted probabilities should be evaluated at the distributional level without being reduced to binary decisions. To this end, we propose a distribution-based evaluation framework that compares model output distributions with reference distributions and measures their divergence using the Wasserstein distance. We further show how discrepancies between model predictions and corpus-observed turn shifts can be used as a basis for training-data refinement. Experiments on Japanese conversational data, using linguistic information alone, showed that the proposed refinement reduced distributional divergence, indicating better alignment between predicted probabilities and the reference distributions. The refinement also improved balanced accuracy in a supplementary binary evaluation.</abstract>
    <identifier type="citekey">kubo-etal-2026-rethinking</identifier>
    <location>
        <url>https://aclanthology.org/2026.sigdial-1.2/</url>
    </location>
    <part>
        <date>2026-08</date>
        <extent unit="page">
            <start>13</start>
            <end>24</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Rethinking Binary Evaluation of Turn-Taking under Inherent Ambiguity
%A Kubo, Yunosuke
%A Yamamoto, Kenta
%A Takeda, Ryu
%A Komatani, Kazunori
%Y Choi, Jinho D.
%Y Chen, Yun-Nung
%Y Funakoshi, Kotaro
%Y Emami, Ali
%S Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue
%D 2026
%8 August
%I Association for Computational Linguistics
%C Atlanta, Georgia, USA
%F kubo-etal-2026-rethinking
%X Turn-taking prediction models output probabilities of turn shifts, yet they are typically evaluated by thresholding these probabilities into binary decisions and comparing them against corpus-observed labels. This practice implicitly treats corpus-observed turn shifts as definitive ground truth, even though under inherent turn-taking ambiguity they reflect one realized interactional outcome among multiple plausible outcomes, rather than a uniquely correct binary label. We argue that binary evaluation is a practical simplification rather than a theoretical necessity. Instead, predicted probabilities should be evaluated at the distributional level without being reduced to binary decisions. To this end, we propose a distribution-based evaluation framework that compares model output distributions with reference distributions and measures their divergence using the Wasserstein distance. We further show how discrepancies between model predictions and corpus-observed turn shifts can be used as a basis for training-data refinement. Experiments on Japanese conversational data, using linguistic information alone, showed that the proposed refinement reduced distributional divergence, indicating better alignment between predicted probabilities and the reference distributions. The refinement also improved balanced accuracy in a supplementary binary evaluation.
%U https://aclanthology.org/2026.sigdial-1.2/
%P 13-24

Markdown (Informal)

[Rethinking Binary Evaluation of Turn-Taking under Inherent Ambiguity](https://aclanthology.org/2026.sigdial-1.2/) (Kubo et al., SIGDIAL 2026)

• Rethinking Binary Evaluation of Turn-Taking under Inherent Ambiguity (Kubo et al., SIGDIAL 2026)

ACL

• Yunosuke Kubo, Kenta Yamamoto, Ryu Takeda, and Kazunori Komatani. 2026. Rethinking Binary Evaluation of Turn-Taking under Inherent Ambiguity. In Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 13–24, Atlanta, Georgia, USA. Association for Computational Linguistics.