<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="nguyen-2018-comparing">
    <titleInfo>
        <title>Comparing Automatic and Human Evaluation of Local Explanations for Text Classification</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Dong</namePart>
        <namePart type="family">Nguyen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2018-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Marilyn</namePart>
            <namePart type="family">Walker</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Heng</namePart>
            <namePart type="family">Ji</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Amanda</namePart>
            <namePart type="family">Stent</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">New Orleans, Louisiana</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Text classification models are becoming increasingly complex and opaque, however for many applications it is essential that the models are interpretable. Recently, a variety of approaches have been proposed for generating local explanations. While robust evaluations are needed to drive further progress, so far it is unclear which evaluation approaches are suitable. This paper is a first step towards more robust evaluations of local explanations. We evaluate a variety of local explanation approaches using automatic measures based on word deletion. Furthermore, we show that an evaluation using a crowdsourcing experiment correlates moderately with these automatic measures and that a variety of other factors also impact the human judgements.</abstract>
    <identifier type="citekey">nguyen-2018-comparing</identifier>
    <identifier type="doi">10.18653/v1/N18-1097</identifier>
    <location>
        <url>https://aclanthology.org/N18-1097</url>
    </location>
    <part>
        <date>2018-06</date>
        <extent unit="page">
            <start>1069</start>
            <end>1078</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Comparing Automatic and Human Evaluation of Local Explanations for Text Classification
%A Nguyen, Dong
%Y Walker, Marilyn
%Y Ji, Heng
%Y Stent, Amanda
%S Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)
%D 2018
%8 June
%I Association for Computational Linguistics
%C New Orleans, Louisiana
%F nguyen-2018-comparing
%X Text classification models are becoming increasingly complex and opaque, however for many applications it is essential that the models are interpretable. Recently, a variety of approaches have been proposed for generating local explanations. While robust evaluations are needed to drive further progress, so far it is unclear which evaluation approaches are suitable. This paper is a first step towards more robust evaluations of local explanations. We evaluate a variety of local explanation approaches using automatic measures based on word deletion. Furthermore, we show that an evaluation using a crowdsourcing experiment correlates moderately with these automatic measures and that a variety of other factors also impact the human judgements.
%R 10.18653/v1/N18-1097
%U https://aclanthology.org/N18-1097
%U https://doi.org/10.18653/v1/N18-1097
%P 1069-1078

Markdown (Informal)

[Comparing Automatic and Human Evaluation of Local Explanations for Text Classification](https://aclanthology.org/N18-1097) (Nguyen, NAACL 2018)

• Comparing Automatic and Human Evaluation of Local Explanations for Text Classification (Nguyen, NAACL 2018)

ACL

• Dong Nguyen. 2018. Comparing Automatic and Human Evaluation of Local Explanations for Text Classification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1069–1078, New Orleans, Louisiana. Association for Computational Linguistics.