<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="mersinias-mahowald-2023-generated">
    <titleInfo>
        <title>For Generated Text, Is NLI-Neutral Text the Best Text?</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Michail</namePart>
        <namePart type="family">Mersinias</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kyle</namePart>
        <namePart type="family">Mahowald</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We explore incorporating natural language inference (NLI) into the text generative pipeline by using a pre-trained NLI model to assess whether a generated sentence entails, contradicts, or is neutral to the prompt and preceding text. First, we show that the NLI task is predictive of generation errors made by GPT-3. We use these results to develop an NLI-informed generation procedure for GPT-J. Then, we evaluate these generations by obtaining human annotations on error types and overall quality. We find that an NLI strategy of maximizing entailment improves text generation when the nucleus sampling randomness parameter value is high, while one which maximizes contradiction is in fact productive when the parameter value is low. Overall, though, we demonstrate that an NLI strategy of maximizing the neutral class provides the highest quality of generated text (significantly better than the vanilla generations), regardless of parameter value.</abstract>
    <identifier type="citekey">mersinias-mahowald-2023-generated</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-emnlp.170</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-emnlp.170</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>2596</start>
            <end>2602</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T For Generated Text, Is NLI-Neutral Text the Best Text?
%A Mersinias, Michail
%A Mahowald, Kyle
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Findings of the Association for Computational Linguistics: EMNLP 2023
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F mersinias-mahowald-2023-generated
%X We explore incorporating natural language inference (NLI) into the text generative pipeline by using a pre-trained NLI model to assess whether a generated sentence entails, contradicts, or is neutral to the prompt and preceding text. First, we show that the NLI task is predictive of generation errors made by GPT-3. We use these results to develop an NLI-informed generation procedure for GPT-J. Then, we evaluate these generations by obtaining human annotations on error types and overall quality. We find that an NLI strategy of maximizing entailment improves text generation when the nucleus sampling randomness parameter value is high, while one which maximizes contradiction is in fact productive when the parameter value is low. Overall, though, we demonstrate that an NLI strategy of maximizing the neutral class provides the highest quality of generated text (significantly better than the vanilla generations), regardless of parameter value.
%R 10.18653/v1/2023.findings-emnlp.170
%U https://aclanthology.org/2023.findings-emnlp.170
%U https://doi.org/10.18653/v1/2023.findings-emnlp.170
%P 2596-2602

Markdown (Informal)

[For Generated Text, Is NLI-Neutral Text the Best Text?](https://aclanthology.org/2023.findings-emnlp.170) (Mersinias & Mahowald, Findings 2023)

• For Generated Text, Is NLI-Neutral Text the Best Text? (Mersinias & Mahowald, Findings 2023)

ACL

• Michail Mersinias and Kyle Mahowald. 2023. For Generated Text, Is NLI-Neutral Text the Best Text?. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2596–2602, Singapore. Association for Computational Linguistics.