<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="bhaumik-etal-2023-taskdiff">
    <titleInfo>
        <title>TaskDiff: A Similarity Metric for Task-Oriented Conversations</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Ankita</namePart>
        <namePart type="family">Bhaumik</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Praveen</namePart>
        <namePart type="family">Venkateswaran</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yara</namePart>
        <namePart type="family">Rizk</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Vatche</namePart>
        <namePart type="family">Isahagian</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The popularity of conversational digital assistants has resulted in the availability of large amounts of conversational data which can be utilized for improved user experience and personalized response generation. Building these assistants using popular large language models like ChatGPT also require additional emphasis on prompt engineering and evaluation methods. Textual similarity metrics are a key ingredient for such analysis and evaluations. While many similarity metrics have been proposed in the literature, they have not proven effective for task-oriented conversations as they do not take advantage of unique conversational features. To address this gap, we present TaskDiff, a novel conversational similarity metric that utilizes different dialogue components (utterances, intents, and slots) and their distributions to compute similarity. Extensive experimental evaluation of TaskDiff on a benchmark dataset demonstrates its superior performance and improved robustness over other related approaches.</abstract>
    <identifier type="citekey">bhaumik-etal-2023-taskdiff</identifier>
    <identifier type="doi">10.18653/v1/2023.emnlp-main.1009</identifier>
    <location>
        <url>https://aclanthology.org/2023.emnlp-main.1009</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>16234</start>
            <end>16240</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T TaskDiff: A Similarity Metric for Task-Oriented Conversations
%A Bhaumik, Ankita
%A Venkateswaran, Praveen
%A Rizk, Yara
%A Isahagian, Vatche
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F bhaumik-etal-2023-taskdiff
%X The popularity of conversational digital assistants has resulted in the availability of large amounts of conversational data which can be utilized for improved user experience and personalized response generation. Building these assistants using popular large language models like ChatGPT also require additional emphasis on prompt engineering and evaluation methods. Textual similarity metrics are a key ingredient for such analysis and evaluations. While many similarity metrics have been proposed in the literature, they have not proven effective for task-oriented conversations as they do not take advantage of unique conversational features. To address this gap, we present TaskDiff, a novel conversational similarity metric that utilizes different dialogue components (utterances, intents, and slots) and their distributions to compute similarity. Extensive experimental evaluation of TaskDiff on a benchmark dataset demonstrates its superior performance and improved robustness over other related approaches.
%R 10.18653/v1/2023.emnlp-main.1009
%U https://aclanthology.org/2023.emnlp-main.1009
%U https://doi.org/10.18653/v1/2023.emnlp-main.1009
%P 16234-16240

Markdown (Informal)

[TaskDiff: A Similarity Metric for Task-Oriented Conversations](https://aclanthology.org/2023.emnlp-main.1009) (Bhaumik et al., EMNLP 2023)

• TaskDiff: A Similarity Metric for Task-Oriented Conversations (Bhaumik et al., EMNLP 2023)

ACL

• Ankita Bhaumik, Praveen Venkateswaran, Yara Rizk, and Vatche Isahagian. 2023. TaskDiff: A Similarity Metric for Task-Oriented Conversations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16234–16240, Singapore. Association for Computational Linguistics.