<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lee-lee-2025-limits">
    <titleInfo>
        <title>The Limits of Post-hoc Preference Adaptation: A Case Study on DSTC12 Clustering</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Jihyun</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Gary</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Twelfth Dialog System Technology Challenge</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Behnam</namePart>
            <namePart type="family">Hedayatnia</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vivian</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zhang</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Raghav</namePart>
            <namePart type="family">Gupta</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Michel</namePart>
            <namePart type="family">Galley</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Avignon, France</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-330-2</identifier>
    </relatedItem>
    <abstract>Understanding user intent in dialogue is essential for controllable and coherent conversational AI. In this work, we present a case study on controllable theme induction in dialogue systems using the DSTC12 Track 2 dataset. Our pipeline integrates LLM-based summarization, utterance clustering, and synthetic preference modeling based on should-link and cannot-link predictions. While preference signals offer moderate improvements in cluster refinement, we observe that their effectiveness is significantly constrained by coarse initial clustering. Experiments on the Finance and Insurance domains show that even authentic human labeled preference struggle when initial clusters do not align with human intent. These findings highlight the need to incorporate preference supervision earlier in the pipeline to ensure semantically coherent clustering.</abstract>
    <identifier type="citekey">lee-lee-2025-limits</identifier>
    <location>
        <url>https://aclanthology.org/2025.dstc-1.4/</url>
    </location>
    <part>
        <date>2025-08</date>
        <extent unit="page">
            <start>36</start>
            <end>43</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T The Limits of Post-hoc Preference Adaptation: A Case Study on DSTC12 Clustering
%A Lee, Jihyun
%A Lee, Gary
%Y Hedayatnia, Behnam
%Y Chen, Vivian
%Y Chen, Zhang
%Y Gupta, Raghav
%Y Galley, Michel
%S Proceedings of the Twelfth Dialog System Technology Challenge
%D 2025
%8 August
%I Association for Computational Linguistics
%C Avignon, France
%@ 979-8-89176-330-2
%F lee-lee-2025-limits
%X Understanding user intent in dialogue is essential for controllable and coherent conversational AI. In this work, we present a case study on controllable theme induction in dialogue systems using the DSTC12 Track 2 dataset. Our pipeline integrates LLM-based summarization, utterance clustering, and synthetic preference modeling based on should-link and cannot-link predictions. While preference signals offer moderate improvements in cluster refinement, we observe that their effectiveness is significantly constrained by coarse initial clustering. Experiments on the Finance and Insurance domains show that even authentic human labeled preference struggle when initial clusters do not align with human intent. These findings highlight the need to incorporate preference supervision earlier in the pipeline to ensure semantically coherent clustering.
%U https://aclanthology.org/2025.dstc-1.4/
%P 36-43

Markdown (Informal)

[The Limits of Post-hoc Preference Adaptation: A Case Study on DSTC12 Clustering](https://aclanthology.org/2025.dstc-1.4/) (Lee & Lee, DSTC 2025)

• The Limits of Post-hoc Preference Adaptation: A Case Study on DSTC12 Clustering (Lee & Lee, DSTC 2025)

ACL

• Jihyun Lee and Gary Lee. 2025. The Limits of Post-hoc Preference Adaptation: A Case Study on DSTC12 Clustering. In Proceedings of the Twelfth Dialog System Technology Challenge, pages 36–43, Avignon, France. Association for Computational Linguistics.