<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="naveen-etal-2023-nli">
    <titleInfo>
        <title>NLI to the Rescue: Mapping Entailment Classes to Hallucination Categories in Abstractive Summarization</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Badathala</namePart>
        <namePart type="family">Naveen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Saxena</namePart>
        <namePart type="family">Ashita</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Bhattacharyya</namePart>
        <namePart type="family">Pushpak</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 20th International Conference on Natural Language Processing (ICON)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">D</namePart>
            <namePart type="given">Pawar</namePart>
            <namePart type="family">Jyoti</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lalitha</namePart>
            <namePart type="given">Devi</namePart>
            <namePart type="family">Sobha</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>NLP Association of India (NLPAI)</publisher>
            <place>
                <placeTerm type="text">Goa University, Goa, India</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this paper, we detect hallucinations in summaries generated by abstractive summarization models. We focus on three types of hallucination viz. intrinsic, extrinsic, and nonhallucinated. The method used for detecting hallucination is based on textual entailment. Given a premise and a hypothesis, textual entailment classifies the hypothesis as contradiction, neutral, or entailment. These three classes of textual entailment are mapped to intrinsic, extrinsic, and non-hallucinated respectively. We fine-tune a RoBERTa-large model on NLI datasets and use it to detect hallucinations on the XSumFaith dataset. We demonstrate that our simple approach using textual entailment outperforms the existing factuality inconsistency detection systems by 12% and we provide insightful analysis of all types of hallucination. To advance research in this area, we create and release a dataset, XSumFaith++, which contains balanced instances of hallucinated and non-hallucinated summaries.</abstract>
    <identifier type="citekey">naveen-etal-2023-nli</identifier>
    <location>
        <url>https://aclanthology.org/2023.icon-1.12</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>120</start>
            <end>132</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T NLI to the Rescue: Mapping Entailment Classes to Hallucination Categories in Abstractive Summarization
%A Naveen, Badathala
%A Ashita, Saxena
%A Pushpak, Bhattacharyya
%Y Jyoti, D. Pawar
%Y Sobha, Lalitha Devi
%S Proceedings of the 20th International Conference on Natural Language Processing (ICON)
%D 2023
%8 December
%I NLP Association of India (NLPAI)
%C Goa University, Goa, India
%F naveen-etal-2023-nli
%X In this paper, we detect hallucinations in summaries generated by abstractive summarization models. We focus on three types of hallucination viz. intrinsic, extrinsic, and nonhallucinated. The method used for detecting hallucination is based on textual entailment. Given a premise and a hypothesis, textual entailment classifies the hypothesis as contradiction, neutral, or entailment. These three classes of textual entailment are mapped to intrinsic, extrinsic, and non-hallucinated respectively. We fine-tune a RoBERTa-large model on NLI datasets and use it to detect hallucinations on the XSumFaith dataset. We demonstrate that our simple approach using textual entailment outperforms the existing factuality inconsistency detection systems by 12% and we provide insightful analysis of all types of hallucination. To advance research in this area, we create and release a dataset, XSumFaith++, which contains balanced instances of hallucinated and non-hallucinated summaries.
%U https://aclanthology.org/2023.icon-1.12
%P 120-132

Markdown (Informal)

[NLI to the Rescue: Mapping Entailment Classes to Hallucination Categories in Abstractive Summarization](https://aclanthology.org/2023.icon-1.12) (Naveen et al., ICON 2023)

• NLI to the Rescue: Mapping Entailment Classes to Hallucination Categories in Abstractive Summarization (Naveen et al., ICON 2023)

ACL

• Badathala Naveen, Saxena Ashita, and Bhattacharyya Pushpak. 2023. NLI to the Rescue: Mapping Entailment Classes to Hallucination Categories in Abstractive Summarization. In Proceedings of the 20th International Conference on Natural Language Processing (ICON), pages 120–132, Goa University, Goa, India. NLP Association of India (NLPAI).