<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="mcguire-tomuro-2021-relation">
    <titleInfo>
        <title>Relation Classification with Cognitive Attention Supervision</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Erik</namePart>
        <namePart type="family">McGuire</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Noriko</namePart>
        <namePart type="family">Tomuro</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Emmanuele</namePart>
            <namePart type="family">Chersoni</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nora</namePart>
            <namePart type="family">Hollenstein</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Cassandra</namePart>
            <namePart type="family">Jacobs</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yohei</namePart>
            <namePart type="family">Oseki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Laurent</namePart>
            <namePart type="family">Prévot</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Enrico</namePart>
            <namePart type="family">Santus</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Many current language models such as BERT utilize attention mechanisms to transform sequence representations. We ask whether we can influence BERT’s attention with human reading patterns by using eye-tracking and brain imaging data. We fine-tune BERT for relation extraction with auxiliary attention supervision in which BERT’s attention weights are supervised by cognitive data. Through a variety of metrics we find that this attention supervision can be used to increase similarity between model attention distributions over sequences and the cognitive data without significantly affecting classification performance while making unique errors from the baseline. In particular, models with cognitive attention supervision more often correctly classified samples misclassified by the baseline.</abstract>
    <identifier type="citekey">mcguire-tomuro-2021-relation</identifier>
    <identifier type="doi">10.18653/v1/2021.cmcl-1.26</identifier>
    <location>
        <url>https://aclanthology.org/2021.cmcl-1.26</url>
    </location>
    <part>
        <date>2021-06</date>
        <extent unit="page">
            <start>222</start>
            <end>232</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Relation Classification with Cognitive Attention Supervision
%A McGuire, Erik
%A Tomuro, Noriko
%Y Chersoni, Emmanuele
%Y Hollenstein, Nora
%Y Jacobs, Cassandra
%Y Oseki, Yohei
%Y Prévot, Laurent
%Y Santus, Enrico
%S Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics
%D 2021
%8 June
%I Association for Computational Linguistics
%C Online
%F mcguire-tomuro-2021-relation
%X Many current language models such as BERT utilize attention mechanisms to transform sequence representations. We ask whether we can influence BERT’s attention with human reading patterns by using eye-tracking and brain imaging data. We fine-tune BERT for relation extraction with auxiliary attention supervision in which BERT’s attention weights are supervised by cognitive data. Through a variety of metrics we find that this attention supervision can be used to increase similarity between model attention distributions over sequences and the cognitive data without significantly affecting classification performance while making unique errors from the baseline. In particular, models with cognitive attention supervision more often correctly classified samples misclassified by the baseline.
%R 10.18653/v1/2021.cmcl-1.26
%U https://aclanthology.org/2021.cmcl-1.26
%U https://doi.org/10.18653/v1/2021.cmcl-1.26
%P 222-232

Markdown (Informal)

[Relation Classification with Cognitive Attention Supervision](https://aclanthology.org/2021.cmcl-1.26) (McGuire & Tomuro, CMCL 2021)

• Relation Classification with Cognitive Attention Supervision (McGuire & Tomuro, CMCL 2021)

ACL

• Erik McGuire and Noriko Tomuro. 2021. Relation Classification with Cognitive Attention Supervision. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pages 222–232, Online. Association for Computational Linguistics.