<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="klein-nabi-2019-attention">
    <titleInfo>
        <title>Attention Is (not) All You Need for Commonsense Reasoning</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Tassilo</namePart>
        <namePart type="family">Klein</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Moin</namePart>
        <namePart type="family">Nabi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Korhonen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">David</namePart>
            <namePart type="family">Traum</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lluís</namePart>
            <namePart type="family">Màrquez</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Florence, Italy</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The recently introduced BERT model exhibits strong performance on several language understanding benchmarks. In this paper, we describe a simple re-implementation of BERT for commonsense reasoning. We show that the attentions produced by BERT can be directly utilized for tasks such as the Pronoun Disambiguation Problem and Winograd Schema Challenge. Our proposed attention-guided commonsense reasoning method is conceptually simple yet empirically powerful. Experimental analysis on multiple datasets demonstrates that our proposed system performs remarkably well on all cases while outperforming the previously reported state of the art by a margin. While results suggest that BERT seems to implicitly learn to establish complex relationships between entities, solving commonsense reasoning tasks might require more than unsupervised models learned from huge text corpora.</abstract>
    <identifier type="citekey">klein-nabi-2019-attention</identifier>
    <identifier type="doi">10.18653/v1/P19-1477</identifier>
    <location>
        <url>https://aclanthology.org/P19-1477</url>
    </location>
    <part>
        <date>2019-07</date>
        <extent unit="page">
            <start>4831</start>
            <end>4836</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Attention Is (not) All You Need for Commonsense Reasoning
%A Klein, Tassilo
%A Nabi, Moin
%Y Korhonen, Anna
%Y Traum, David
%Y Màrquez, Lluís
%S Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
%D 2019
%8 July
%I Association for Computational Linguistics
%C Florence, Italy
%F klein-nabi-2019-attention
%X The recently introduced BERT model exhibits strong performance on several language understanding benchmarks. In this paper, we describe a simple re-implementation of BERT for commonsense reasoning. We show that the attentions produced by BERT can be directly utilized for tasks such as the Pronoun Disambiguation Problem and Winograd Schema Challenge. Our proposed attention-guided commonsense reasoning method is conceptually simple yet empirically powerful. Experimental analysis on multiple datasets demonstrates that our proposed system performs remarkably well on all cases while outperforming the previously reported state of the art by a margin. While results suggest that BERT seems to implicitly learn to establish complex relationships between entities, solving commonsense reasoning tasks might require more than unsupervised models learned from huge text corpora.
%R 10.18653/v1/P19-1477
%U https://aclanthology.org/P19-1477
%U https://doi.org/10.18653/v1/P19-1477
%P 4831-4836

Markdown (Informal)

[Attention Is (not) All You Need for Commonsense Reasoning](https://aclanthology.org/P19-1477) (Klein & Nabi, ACL 2019)

• Attention Is (not) All You Need for Commonsense Reasoning (Klein & Nabi, ACL 2019)

ACL

• Tassilo Klein and Moin Nabi. 2019. Attention Is (not) All You Need for Commonsense Reasoning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4831–4836, Florence, Italy. Association for Computational Linguistics.