<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="jiang-marneffe-2022-investigating">
    <titleInfo>
        <title>Investigating Reasons for Disagreement in Natural Language Inference</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Nan-Jiang</namePart>
        <namePart type="family">Jiang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Marie-Catherine</namePart>
        <namePart type="family">de Marneffe</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>We investigate how disagreement in natural language inference (NLI) annotation arises. We developed a taxonomy of disagreement sources with 10 categories spanning 3 high- level classes. We found that some disagreements are due to uncertainty in the sentence meaning, others to annotator biases and task artifacts, leading to different interpretations of the label distribution. We explore two modeling approaches for detecting items with potential disagreement: a 4-way classification with a “Complicated” label in addition to the three standard NLI labels, and a multilabel classification approach. We found that the multilabel classification is more expressive and gives better recall of the possible interpretations in the data.</abstract>
    <identifier type="citekey">jiang-marneffe-2022-investigating</identifier>
    <identifier type="doi">10.1162/tacl_a_00523</identifier>
    <location>
        <url>https://aclanthology.org/2022.tacl-1.78</url>
    </location>
    <part>
        <date>2022</date>
        <detail type="volume"><number>10</number></detail>
        <extent unit="page">
            <start>1357</start>
            <end>1374</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T Investigating Reasons for Disagreement in Natural Language Inference
%A Jiang, Nan-Jiang
%A de Marneffe, Marie-Catherine
%J Transactions of the Association for Computational Linguistics
%D 2022
%V 10
%I MIT Press
%C Cambridge, MA
%F jiang-marneffe-2022-investigating
%X We investigate how disagreement in natural language inference (NLI) annotation arises. We developed a taxonomy of disagreement sources with 10 categories spanning 3 high- level classes. We found that some disagreements are due to uncertainty in the sentence meaning, others to annotator biases and task artifacts, leading to different interpretations of the label distribution. We explore two modeling approaches for detecting items with potential disagreement: a 4-way classification with a “Complicated” label in addition to the three standard NLI labels, and a multilabel classification approach. We found that the multilabel classification is more expressive and gives better recall of the possible interpretations in the data.
%R 10.1162/tacl_a_00523
%U https://aclanthology.org/2022.tacl-1.78
%U https://doi.org/10.1162/tacl_a_00523
%P 1357-1374

Markdown (Informal)

[Investigating Reasons for Disagreement in Natural Language Inference](https://aclanthology.org/2022.tacl-1.78) (Jiang & de Marneffe, TACL 2022)

• Investigating Reasons for Disagreement in Natural Language Inference (Jiang & de Marneffe, TACL 2022)

ACL

• Nan-Jiang Jiang and Marie-Catherine de Marneffe. 2022. Investigating Reasons for Disagreement in Natural Language Inference. Transactions of the Association for Computational Linguistics, 10:1357–1374.