<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yin-schutze-2018-attentive">
    <titleInfo>
        <title>Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Wenpeng</namePart>
        <namePart type="family">Yin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hinrich</namePart>
        <namePart type="family">Schütze</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2018</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>In NLP, convolutional neural networks (CNNs) have benefited less than recurrent neural networks (RNNs) from attention mechanisms. We hypothesize that this is because the attention in CNNs has been mainly implemented as attentive pooling (i.e., it is applied to pooling) rather than as attentive convolution (i.e., it is integrated into convolution). Convolution is the differentiator of CNNs in that it can powerfully model the higher-level representation of a word by taking into account its local fixed-size context in the input text tx. In this work, we propose an attentive convolution network, ATTCONV. It extends the context scope of the convolution operation, deriving higher-level features for a word not only from local context, but also from information extracted from nonlocal context by the attention mechanism commonly used in RNNs. This nonlocal context can come (i) from parts of the input text tx that are distant or (ii) from extra (i.e., external) contexts ty. Experiments on sentence modeling with zero-context (sentiment analysis), single-context (textual entailment) and multiple-context (claim verification) demonstrate the effectiveness of ATTCONV in sentence representation learning with the incorporation of context. In particular, attentive convolution outperforms attentive pooling and is a strong competitor to popular attentive RNNs.1</abstract>
    <identifier type="citekey">yin-schutze-2018-attentive</identifier>
    <identifier type="doi">10.1162/tacl_a_00249</identifier>
    <location>
        <url>https://aclanthology.org/Q18-1047</url>
    </location>
    <part>
        <date>2018</date>
        <detail type="volume"><number>6</number></detail>
        <extent unit="page">
            <start>687</start>
            <end>702</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms
%A Yin, Wenpeng
%A Schütze, Hinrich
%J Transactions of the Association for Computational Linguistics
%D 2018
%V 6
%I MIT Press
%C Cambridge, MA
%F yin-schutze-2018-attentive
%X In NLP, convolutional neural networks (CNNs) have benefited less than recurrent neural networks (RNNs) from attention mechanisms. We hypothesize that this is because the attention in CNNs has been mainly implemented as attentive pooling (i.e., it is applied to pooling) rather than as attentive convolution (i.e., it is integrated into convolution). Convolution is the differentiator of CNNs in that it can powerfully model the higher-level representation of a word by taking into account its local fixed-size context in the input text tx. In this work, we propose an attentive convolution network, ATTCONV. It extends the context scope of the convolution operation, deriving higher-level features for a word not only from local context, but also from information extracted from nonlocal context by the attention mechanism commonly used in RNNs. This nonlocal context can come (i) from parts of the input text tx that are distant or (ii) from extra (i.e., external) contexts ty. Experiments on sentence modeling with zero-context (sentiment analysis), single-context (textual entailment) and multiple-context (claim verification) demonstrate the effectiveness of ATTCONV in sentence representation learning with the incorporation of context. In particular, attentive convolution outperforms attentive pooling and is a strong competitor to popular attentive RNNs.1
%R 10.1162/tacl_a_00249
%U https://aclanthology.org/Q18-1047
%U https://doi.org/10.1162/tacl_a_00249
%P 687-702

Markdown (Informal)

[Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms](https://aclanthology.org/Q18-1047) (Yin & Schütze, TACL 2018)

• Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms (Yin & Schütze, TACL 2018)

ACL

• Wenpeng Yin and Hinrich Schütze. 2018. Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms. Transactions of the Association for Computational Linguistics, 6:687–702.