<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="aoyama-schneider-2022-probe">
    <titleInfo>
        <title>Probe-Less Probing of BERT’s Layer-Wise Linguistic Knowledge with Masked Word Prediction</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Tatsuya</namePart>
        <namePart type="family">Aoyama</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nathan</namePart>
        <namePart type="family">Schneider</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Daphne</namePart>
            <namePart type="family">Ippolito</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Liunian</namePart>
            <namePart type="given">Harold</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Maria</namePart>
            <namePart type="given">Leonor</namePart>
            <namePart type="family">Pacheco</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Danqi</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nianwen</namePart>
            <namePart type="family">Xue</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Hybrid: Seattle, Washington + Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The current study quantitatively (and qualitatively for an illustrative purpose) analyzes BERT’s layer-wise masked word prediction on an English corpus, and finds that (1) the layerwise localization of linguistic knowledge primarily shown in probing studies is replicated in a behavior-based design and (2) that syntactic and semantic information is encoded at different layers for words of different syntactic categories. Hypothesizing that the above results are correlated with the number of likely potential candidates of the masked word prediction, we also investigate how the results differ for tokens within multiword expressions.</abstract>
    <identifier type="citekey">aoyama-schneider-2022-probe</identifier>
    <identifier type="doi">10.18653/v1/2022.naacl-srw.25</identifier>
    <location>
        <url>https://aclanthology.org/2022.naacl-srw.25</url>
    </location>
    <part>
        <date>2022-07</date>
        <extent unit="page">
            <start>195</start>
            <end>201</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Probe-Less Probing of BERT’s Layer-Wise Linguistic Knowledge with Masked Word Prediction
%A Aoyama, Tatsuya
%A Schneider, Nathan
%Y Ippolito, Daphne
%Y Li, Liunian Harold
%Y Pacheco, Maria Leonor
%Y Chen, Danqi
%Y Xue, Nianwen
%S Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop
%D 2022
%8 July
%I Association for Computational Linguistics
%C Hybrid: Seattle, Washington + Online
%F aoyama-schneider-2022-probe
%X The current study quantitatively (and qualitatively for an illustrative purpose) analyzes BERT’s layer-wise masked word prediction on an English corpus, and finds that (1) the layerwise localization of linguistic knowledge primarily shown in probing studies is replicated in a behavior-based design and (2) that syntactic and semantic information is encoded at different layers for words of different syntactic categories. Hypothesizing that the above results are correlated with the number of likely potential candidates of the masked word prediction, we also investigate how the results differ for tokens within multiword expressions.
%R 10.18653/v1/2022.naacl-srw.25
%U https://aclanthology.org/2022.naacl-srw.25
%U https://doi.org/10.18653/v1/2022.naacl-srw.25
%P 195-201

Markdown (Informal)

[Probe-Less Probing of BERT’s Layer-Wise Linguistic Knowledge with Masked Word Prediction](https://aclanthology.org/2022.naacl-srw.25) (Aoyama & Schneider, NAACL 2022)

• Probe-Less Probing of BERT’s Layer-Wise Linguistic Knowledge with Masked Word Prediction (Aoyama & Schneider, NAACL 2022)

ACL

• Tatsuya Aoyama and Nathan Schneider. 2022. Probe-Less Probing of BERT’s Layer-Wise Linguistic Knowledge with Masked Word Prediction. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop, pages 195–201, Hybrid: Seattle, Washington + Online. Association for Computational Linguistics.