<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="derby-etal-2021-representation">
    <titleInfo>
        <title>Representation and Pre-Activation of Lexical-Semantic Knowledge in Neural Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Steven</namePart>
        <namePart type="family">Derby</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Paul</namePart>
        <namePart type="family">Miller</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Barry</namePart>
        <namePart type="family">Devereux</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Emmanuele</namePart>
            <namePart type="family">Chersoni</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nora</namePart>
            <namePart type="family">Hollenstein</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Cassandra</namePart>
            <namePart type="family">Jacobs</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yohei</namePart>
            <namePart type="family">Oseki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Laurent</namePart>
            <namePart type="family">Prévot</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Enrico</namePart>
            <namePart type="family">Santus</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this paper, we perform a systematic analysis of how closely the intermediate layers from LSTM and trans former language models correspond to human semantic knowledge. Furthermore, in order to make more meaningful comparisons with theories of human language comprehension in psycholinguistics, we focus on two key stages where the meaning of a particular target word may arise: immediately before the word’s presentation to the model (comparable to forward inferencing), and immediately after the word token has been input into the network. Our results indicate that the transformer models are better at capturing semantic knowledge relating to lexical concepts, both during word prediction and when retention is required.</abstract>
    <identifier type="citekey">derby-etal-2021-representation</identifier>
    <identifier type="doi">10.18653/v1/2021.cmcl-1.25</identifier>
    <location>
        <url>https://aclanthology.org/2021.cmcl-1.25</url>
    </location>
    <part>
        <date>2021-06</date>
        <extent unit="page">
            <start>211</start>
            <end>221</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Representation and Pre-Activation of Lexical-Semantic Knowledge in Neural Language Models
%A Derby, Steven
%A Miller, Paul
%A Devereux, Barry
%Y Chersoni, Emmanuele
%Y Hollenstein, Nora
%Y Jacobs, Cassandra
%Y Oseki, Yohei
%Y Prévot, Laurent
%Y Santus, Enrico
%S Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics
%D 2021
%8 June
%I Association for Computational Linguistics
%C Online
%F derby-etal-2021-representation
%X In this paper, we perform a systematic analysis of how closely the intermediate layers from LSTM and trans former language models correspond to human semantic knowledge. Furthermore, in order to make more meaningful comparisons with theories of human language comprehension in psycholinguistics, we focus on two key stages where the meaning of a particular target word may arise: immediately before the word’s presentation to the model (comparable to forward inferencing), and immediately after the word token has been input into the network. Our results indicate that the transformer models are better at capturing semantic knowledge relating to lexical concepts, both during word prediction and when retention is required.
%R 10.18653/v1/2021.cmcl-1.25
%U https://aclanthology.org/2021.cmcl-1.25
%U https://doi.org/10.18653/v1/2021.cmcl-1.25
%P 211-221

Markdown (Informal)

[Representation and Pre-Activation of Lexical-Semantic Knowledge in Neural Language Models](https://aclanthology.org/2021.cmcl-1.25) (Derby et al., CMCL 2021)

• Representation and Pre-Activation of Lexical-Semantic Knowledge in Neural Language Models (Derby et al., CMCL 2021)

ACL

• Steven Derby, Paul Miller, and Barry Devereux. 2021. Representation and Pre-Activation of Lexical-Semantic Knowledge in Neural Language Models. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pages 211–221, Online. Association for Computational Linguistics.