<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kamp-etal-2023-dynamic">
    <titleInfo>
        <title>Dynamic Top-k Estimation Consolidates Disagreement between Feature Attribution Methods</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Jonathan</namePart>
        <namePart type="family">Kamp</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lisa</namePart>
        <namePart type="family">Beinborn</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Antske</namePart>
        <namePart type="family">Fokkens</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Feature attribution scores are used for explaining the prediction of a text classifier to users by highlighting a k number of tokens. In this work, we propose a way to determine the number of optimal k tokens that should be displayed from sequential properties of the attribution scores. Our approach is dynamic across sentences, method-agnostic, and deals with sentence length bias. We compare agreement between multiple methods and humans on an NLI task, using fixed k and dynamic k. We find that perturbation-based methods and Vanilla Gradient exhibit highest agreement on most method–method and method–human agreement metrics with a static k. Their advantage over other methods disappears with dynamic ks which mainly improve Integrated Gradient and GradientXInput. To our knowledge, this is the first evidence that sequential properties of attribution scores are informative for consolidating attribution signals for human interpretation.</abstract>
    <identifier type="citekey">kamp-etal-2023-dynamic</identifier>
    <identifier type="doi">10.18653/v1/2023.emnlp-main.379</identifier>
    <location>
        <url>https://aclanthology.org/2023.emnlp-main.379</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>6190</start>
            <end>6197</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Dynamic Top-k Estimation Consolidates Disagreement between Feature Attribution Methods
%A Kamp, Jonathan
%A Beinborn, Lisa
%A Fokkens, Antske
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F kamp-etal-2023-dynamic
%X Feature attribution scores are used for explaining the prediction of a text classifier to users by highlighting a k number of tokens. In this work, we propose a way to determine the number of optimal k tokens that should be displayed from sequential properties of the attribution scores. Our approach is dynamic across sentences, method-agnostic, and deals with sentence length bias. We compare agreement between multiple methods and humans on an NLI task, using fixed k and dynamic k. We find that perturbation-based methods and Vanilla Gradient exhibit highest agreement on most method–method and method–human agreement metrics with a static k. Their advantage over other methods disappears with dynamic ks which mainly improve Integrated Gradient and GradientXInput. To our knowledge, this is the first evidence that sequential properties of attribution scores are informative for consolidating attribution signals for human interpretation.
%R 10.18653/v1/2023.emnlp-main.379
%U https://aclanthology.org/2023.emnlp-main.379
%U https://doi.org/10.18653/v1/2023.emnlp-main.379
%P 6190-6197

Markdown (Informal)

[Dynamic Top-k Estimation Consolidates Disagreement between Feature Attribution Methods](https://aclanthology.org/2023.emnlp-main.379) (Kamp et al., EMNLP 2023)

• Dynamic Top-k Estimation Consolidates Disagreement between Feature Attribution Methods (Kamp et al., EMNLP 2023)

ACL

• Jonathan Kamp, Lisa Beinborn, and Antske Fokkens. 2023. Dynamic Top-k Estimation Consolidates Disagreement between Feature Attribution Methods. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6190–6197, Singapore. Association for Computational Linguistics.