<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="akani-etal-2023-reducing">
    <titleInfo>
        <title>Reducing named entity hallucination risk to ensure faithful summary generation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Eunice</namePart>
        <namePart type="family">Akani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Benoit</namePart>
        <namePart type="family">Favre</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Frederic</namePart>
        <namePart type="family">Bechet</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Romain</namePart>
        <namePart type="family">Gemignani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 16th International Natural Language Generation Conference</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">C</namePart>
            <namePart type="given">Maria</namePart>
            <namePart type="family">Keet</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sina</namePart>
            <namePart type="family">Zarrieß</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Prague, Czechia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The faithfulness of abstractive text summarization at the named entities level is the focus of this study. We propose to add a new criterion to the summary selection method based on the “risk” of generating entities that do not belong to the source document. This method is based on the assumption that Out-Of-Document entities are more likely to be hallucinations. This assumption was verified by a manual annotation of the entities occurring in a set of generated summaries on the CNN/DM corpus. This study showed that only 29% of the entities outside the source document were inferrable by the annotators, leading to 71% of hallucinations among OOD entities. We test our selection method on the CNN/DM corpus and show that it significantly reduces the hallucination risk on named entities while maintaining competitive results with respect to automatic evaluation metrics like ROUGE.</abstract>
    <identifier type="citekey">akani-etal-2023-reducing</identifier>
    <identifier type="doi">10.18653/v1/2023.inlg-main.33</identifier>
    <location>
        <url>https://aclanthology.org/2023.inlg-main.33</url>
    </location>
    <part>
        <date>2023-09</date>
        <extent unit="page">
            <start>437</start>
            <end>442</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Reducing named entity hallucination risk to ensure faithful summary generation
%A Akani, Eunice
%A Favre, Benoit
%A Bechet, Frederic
%A Gemignani, Romain
%Y Keet, C. Maria
%Y Lee, Hung-Yi
%Y Zarrieß, Sina
%S Proceedings of the 16th International Natural Language Generation Conference
%D 2023
%8 September
%I Association for Computational Linguistics
%C Prague, Czechia
%F akani-etal-2023-reducing
%X The faithfulness of abstractive text summarization at the named entities level is the focus of this study. We propose to add a new criterion to the summary selection method based on the “risk” of generating entities that do not belong to the source document. This method is based on the assumption that Out-Of-Document entities are more likely to be hallucinations. This assumption was verified by a manual annotation of the entities occurring in a set of generated summaries on the CNN/DM corpus. This study showed that only 29% of the entities outside the source document were inferrable by the annotators, leading to 71% of hallucinations among OOD entities. We test our selection method on the CNN/DM corpus and show that it significantly reduces the hallucination risk on named entities while maintaining competitive results with respect to automatic evaluation metrics like ROUGE.
%R 10.18653/v1/2023.inlg-main.33
%U https://aclanthology.org/2023.inlg-main.33
%U https://doi.org/10.18653/v1/2023.inlg-main.33
%P 437-442

Markdown (Informal)

[Reducing named entity hallucination risk to ensure faithful summary generation](https://aclanthology.org/2023.inlg-main.33) (Akani et al., INLG-SIGDIAL 2023)

• Reducing named entity hallucination risk to ensure faithful summary generation (Akani et al., INLG-SIGDIAL 2023)

ACL

• Eunice Akani, Benoit Favre, Frederic Bechet, and Romain Gemignani. 2023. Reducing named entity hallucination risk to ensure faithful summary generation. In Proceedings of the 16th International Natural Language Generation Conference, pages 437–442, Prague, Czechia. Association for Computational Linguistics.