<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lee-etal-2023-empathy">
    <titleInfo>
        <title>Empathy Identification Systems are not Accurately Accounting for Context</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Andrew</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jonathan</namePart>
        <namePart type="given">K</namePart>
        <namePart type="family">Kummerfeld</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Larry</namePart>
        <namePart type="family">An</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Rada</namePart>
        <namePart type="family">Mihalcea</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Andreas</namePart>
            <namePart type="family">Vlachos</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Isabelle</namePart>
            <namePart type="family">Augenstein</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Dubrovnik, Croatia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Understanding empathy in text dialogue data is a difficult, yet critical, skill for effective human-machine interaction. In this work, we ask whether systems are making meaningful progress on this challenge. We consider a simple model that checks if an input utterance is similar to a small set of empathetic examples. Crucially, the model does not look at what the utterance is a response to, i.e., the dialogue context. This model performs comparably to other work on standard benchmarks and even outperforms state-of-the-art models for empathetic rationale extraction by 16.7 points on T-F1 and 4.3 on IOU-F1. This indicates that current systems rely on the surface form of the response, rather than whether it is suitable in context. To confirm this, we create examples with dialogue contexts that change the interpretation of the response and show that current systems continue to label utterances as empathetic. We discuss the implications of our findings, including improvements for empathetic benchmarks and how our model can be an informative baseline.</abstract>
    <identifier type="citekey">lee-etal-2023-empathy</identifier>
    <identifier type="doi">10.18653/v1/2023.eacl-main.123</identifier>
    <location>
        <url>https://aclanthology.org/2023.eacl-main.123</url>
    </location>
    <part>
        <date>2023-05</date>
        <extent unit="page">
            <start>1686</start>
            <end>1695</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Empathy Identification Systems are not Accurately Accounting for Context
%A Lee, Andrew
%A Kummerfeld, Jonathan K.
%A An, Larry
%A Mihalcea, Rada
%Y Vlachos, Andreas
%Y Augenstein, Isabelle
%S Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
%D 2023
%8 May
%I Association for Computational Linguistics
%C Dubrovnik, Croatia
%F lee-etal-2023-empathy
%X Understanding empathy in text dialogue data is a difficult, yet critical, skill for effective human-machine interaction. In this work, we ask whether systems are making meaningful progress on this challenge. We consider a simple model that checks if an input utterance is similar to a small set of empathetic examples. Crucially, the model does not look at what the utterance is a response to, i.e., the dialogue context. This model performs comparably to other work on standard benchmarks and even outperforms state-of-the-art models for empathetic rationale extraction by 16.7 points on T-F1 and 4.3 on IOU-F1. This indicates that current systems rely on the surface form of the response, rather than whether it is suitable in context. To confirm this, we create examples with dialogue contexts that change the interpretation of the response and show that current systems continue to label utterances as empathetic. We discuss the implications of our findings, including improvements for empathetic benchmarks and how our model can be an informative baseline.
%R 10.18653/v1/2023.eacl-main.123
%U https://aclanthology.org/2023.eacl-main.123
%U https://doi.org/10.18653/v1/2023.eacl-main.123
%P 1686-1695

Markdown (Informal)

[Empathy Identification Systems are not Accurately Accounting for Context](https://aclanthology.org/2023.eacl-main.123) (Lee et al., EACL 2023)

• Empathy Identification Systems are not Accurately Accounting for Context (Lee et al., EACL 2023)

ACL

• Andrew Lee, Jonathan K. Kummerfeld, Larry An, and Rada Mihalcea. 2023. Empathy Identification Systems are not Accurately Accounting for Context. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1686–1695, Dubrovnik, Croatia. Association for Computational Linguistics.