<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="biju-etal-2022-input">
    <titleInfo>
        <title>Input-specific Attention Subnetworks for Adversarial Detection</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Emil</namePart>
        <namePart type="family">Biju</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Anirudh</namePart>
        <namePart type="family">Sriram</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Pratyush</namePart>
        <namePart type="family">Kumar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mitesh</namePart>
        <namePart type="family">Khapra</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2022</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Smaranda</namePart>
            <namePart type="family">Muresan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Preslav</namePart>
            <namePart type="family">Nakov</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Aline</namePart>
            <namePart type="family">Villavicencio</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Dublin, Ireland</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Self-attention heads are characteristic of Transformer models and have been well studied for interpretability and pruning. In this work, we demonstrate an altogether different utility of attention heads, namely for adversarial detection. Specifically, we propose a method to construct input-specific attention subnetworks (IAS) from which we extract three features to discriminate between authentic and adversarial inputs. The resultant detector significantly improves (by over 7.5%) the state-of-the-art adversarial detection accuracy for the BERT encoder on 10 NLU datasets with 11 different adversarial attack types. We also demonstrate that our method (a) is more accurate for larger models which are likely to have more spurious correlations and thus vulnerable to adversarial attack, and (b) performs well even with modest training sets of adversarial examples.</abstract>
    <identifier type="citekey">biju-etal-2022-input</identifier>
    <identifier type="doi">10.18653/v1/2022.findings-acl.4</identifier>
    <location>
        <url>https://aclanthology.org/2022.findings-acl.4</url>
    </location>
    <part>
        <date>2022-05</date>
        <extent unit="page">
            <start>31</start>
            <end>44</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Input-specific Attention Subnetworks for Adversarial Detection
%A Biju, Emil
%A Sriram, Anirudh
%A Kumar, Pratyush
%A Khapra, Mitesh
%Y Muresan, Smaranda
%Y Nakov, Preslav
%Y Villavicencio, Aline
%S Findings of the Association for Computational Linguistics: ACL 2022
%D 2022
%8 May
%I Association for Computational Linguistics
%C Dublin, Ireland
%F biju-etal-2022-input
%X Self-attention heads are characteristic of Transformer models and have been well studied for interpretability and pruning. In this work, we demonstrate an altogether different utility of attention heads, namely for adversarial detection. Specifically, we propose a method to construct input-specific attention subnetworks (IAS) from which we extract three features to discriminate between authentic and adversarial inputs. The resultant detector significantly improves (by over 7.5%) the state-of-the-art adversarial detection accuracy for the BERT encoder on 10 NLU datasets with 11 different adversarial attack types. We also demonstrate that our method (a) is more accurate for larger models which are likely to have more spurious correlations and thus vulnerable to adversarial attack, and (b) performs well even with modest training sets of adversarial examples.
%R 10.18653/v1/2022.findings-acl.4
%U https://aclanthology.org/2022.findings-acl.4
%U https://doi.org/10.18653/v1/2022.findings-acl.4
%P 31-44

Markdown (Informal)

[Input-specific Attention Subnetworks for Adversarial Detection](https://aclanthology.org/2022.findings-acl.4) (Biju et al., Findings 2022)

• Input-specific Attention Subnetworks for Adversarial Detection (Biju et al., Findings 2022)

ACL

• Emil Biju, Anirudh Sriram, Pratyush Kumar, and Mitesh Khapra. 2022. Input-specific Attention Subnetworks for Adversarial Detection. In Findings of the Association for Computational Linguistics: ACL 2022, pages 31–44, Dublin, Ireland. Association for Computational Linguistics.