<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="hidey-sarthak-2023-compute">
    <titleInfo>
        <title>Compute-Efficient Churn Reduction for Conversational Agents</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Christopher</namePart>
        <namePart type="family">Hidey</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sarthak</namePart>
        <namePart type="family">Sarthak</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Mingxuan</namePart>
            <namePart type="family">Wang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Imed</namePart>
            <namePart type="family">Zitouni</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Model churn occurs when re-training a model yields different predictions despite using the same data and hyper-parameters. Churn reduction is crucial for industry conversational systems where users expect consistent results for the same queries. In this setting, compute resources are often limited due to latency requirements during serving and overall time constraints during re-training. To address this issue, we propose a compute-efficient method that mitigates churn without requiring extra resources for training or inference. Our approach involves a lightweight data pre-processing step that pairs semantic parses based on their “function call signature” and encourages similarity through an additional loss based on Jensen-Shannon Divergence. We validate the effectiveness of our method in three scenarios: academic (+3.93 percent improvement on average in a churn reduction metric), simulated noisy data (+8.09), and industry (+5.28) settings.</abstract>
    <identifier type="citekey">hidey-sarthak-2023-compute</identifier>
    <identifier type="doi">10.18653/v1/2023.emnlp-industry.28</identifier>
    <location>
        <url>https://aclanthology.org/2023.emnlp-industry.28</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>284</start>
            <end>293</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Compute-Efficient Churn Reduction for Conversational Agents
%A Hidey, Christopher
%A Sarthak, Sarthak
%Y Wang, Mingxuan
%Y Zitouni, Imed
%S Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F hidey-sarthak-2023-compute
%X Model churn occurs when re-training a model yields different predictions despite using the same data and hyper-parameters. Churn reduction is crucial for industry conversational systems where users expect consistent results for the same queries. In this setting, compute resources are often limited due to latency requirements during serving and overall time constraints during re-training. To address this issue, we propose a compute-efficient method that mitigates churn without requiring extra resources for training or inference. Our approach involves a lightweight data pre-processing step that pairs semantic parses based on their “function call signature” and encourages similarity through an additional loss based on Jensen-Shannon Divergence. We validate the effectiveness of our method in three scenarios: academic (+3.93 percent improvement on average in a churn reduction metric), simulated noisy data (+8.09), and industry (+5.28) settings.
%R 10.18653/v1/2023.emnlp-industry.28
%U https://aclanthology.org/2023.emnlp-industry.28
%U https://doi.org/10.18653/v1/2023.emnlp-industry.28
%P 284-293

Markdown (Informal)

[Compute-Efficient Churn Reduction for Conversational Agents](https://aclanthology.org/2023.emnlp-industry.28) (Hidey & Sarthak, EMNLP 2023)

• Compute-Efficient Churn Reduction for Conversational Agents (Hidey & Sarthak, EMNLP 2023)

ACL

• Christopher Hidey and Sarthak Sarthak. 2023. Compute-Efficient Churn Reduction for Conversational Agents. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 284–293, Singapore. Association for Computational Linguistics.