<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="goyal-etal-2023-factual">
    <titleInfo>
        <title>Factual or Contextual? Disentangling Error Types in Entity Description Generation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Navita</namePart>
        <namePart type="family">Goyal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ani</namePart>
        <namePart type="family">Nenkova</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hal</namePart>
        <namePart type="family">Daumé III</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jordan</namePart>
            <namePart type="family">Boyd-Graber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Naoaki</namePart>
            <namePart type="family">Okazaki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In the task of entity description generation, given a context and a specified entity, a model must describe that entity correctly and in a contextually-relevant way. In this task, as well as broader language generation tasks, the generation of a nonfactual description (factual error) versus an incongruous description (contextual error) is fundamentally different, yet often conflated. We develop an evaluation paradigm that enables us to disentangle these two types of errors in naturally occurring textual contexts. We find that factuality and congruity are often at odds, and that models specifically struggle with accurate descriptions of entities that are less familiar to people. This shortcoming of language models raises concerns around the trustworthiness of such models, since factual errors on less well-known entities are exactly those that a human reader will not recognize.</abstract>
    <identifier type="citekey">goyal-etal-2023-factual</identifier>
    <identifier type="doi">10.18653/v1/2023.acl-long.463</identifier>
    <location>
        <url>https://aclanthology.org/2023.acl-long.463</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>8322</start>
            <end>8340</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Factual or Contextual? Disentangling Error Types in Entity Description Generation
%A Goyal, Navita
%A Nenkova, Ani
%A Daumé III, Hal
%Y Rogers, Anna
%Y Boyd-Graber, Jordan
%Y Okazaki, Naoaki
%S Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F goyal-etal-2023-factual
%X In the task of entity description generation, given a context and a specified entity, a model must describe that entity correctly and in a contextually-relevant way. In this task, as well as broader language generation tasks, the generation of a nonfactual description (factual error) versus an incongruous description (contextual error) is fundamentally different, yet often conflated. We develop an evaluation paradigm that enables us to disentangle these two types of errors in naturally occurring textual contexts. We find that factuality and congruity are often at odds, and that models specifically struggle with accurate descriptions of entities that are less familiar to people. This shortcoming of language models raises concerns around the trustworthiness of such models, since factual errors on less well-known entities are exactly those that a human reader will not recognize.
%R 10.18653/v1/2023.acl-long.463
%U https://aclanthology.org/2023.acl-long.463
%U https://doi.org/10.18653/v1/2023.acl-long.463
%P 8322-8340

Markdown (Informal)

[Factual or Contextual? Disentangling Error Types in Entity Description Generation](https://aclanthology.org/2023.acl-long.463) (Goyal et al., ACL 2023)

• Factual or Contextual? Disentangling Error Types in Entity Description Generation (Goyal et al., ACL 2023)

ACL

• Navita Goyal, Ani Nenkova, and Hal Daumé III. 2023. Factual or Contextual? Disentangling Error Types in Entity Description Generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8322–8340, Toronto, Canada. Association for Computational Linguistics.