<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="ren-zhu-2022-specializing">
    <titleInfo>
        <title>Specializing Pre-trained Language Models for Better Relational Reasoning via Network Pruning</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Siyu</namePart>
        <namePart type="family">Ren</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kenny</namePart>
        <namePart type="family">Zhu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: NAACL 2022</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Marine</namePart>
            <namePart type="family">Carpuat</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marie-Catherine</namePart>
            <namePart type="family">de Marneffe</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ivan</namePart>
            <namePart type="given">Vladimir</namePart>
            <namePart type="family">Meza Ruiz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Seattle, United States</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Pretrained masked language models (PLMs) were shown to be inheriting a considerable amount of relational knowledge from the source corpora. In this paper, we present an in-depth and comprehensive study concerning specializing PLMs into relational models from the perspective of network pruning. We show that it is possible to find subnetworks capable of representing grounded commonsense relations at non-trivial sparsity while being more generalizable than original PLMs in scenarios requiring knowledge of single or multiple commonsense relations.</abstract>
    <identifier type="citekey">ren-zhu-2022-specializing</identifier>
    <identifier type="doi">10.18653/v1/2022.findings-naacl.169</identifier>
    <location>
        <url>https://aclanthology.org/2022.findings-naacl.169</url>
    </location>
    <part>
        <date>2022-07</date>
        <extent unit="page">
            <start>2195</start>
            <end>2207</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Specializing Pre-trained Language Models for Better Relational Reasoning via Network Pruning
%A Ren, Siyu
%A Zhu, Kenny
%Y Carpuat, Marine
%Y de Marneffe, Marie-Catherine
%Y Meza Ruiz, Ivan Vladimir
%S Findings of the Association for Computational Linguistics: NAACL 2022
%D 2022
%8 July
%I Association for Computational Linguistics
%C Seattle, United States
%F ren-zhu-2022-specializing
%X Pretrained masked language models (PLMs) were shown to be inheriting a considerable amount of relational knowledge from the source corpora. In this paper, we present an in-depth and comprehensive study concerning specializing PLMs into relational models from the perspective of network pruning. We show that it is possible to find subnetworks capable of representing grounded commonsense relations at non-trivial sparsity while being more generalizable than original PLMs in scenarios requiring knowledge of single or multiple commonsense relations.
%R 10.18653/v1/2022.findings-naacl.169
%U https://aclanthology.org/2022.findings-naacl.169
%U https://doi.org/10.18653/v1/2022.findings-naacl.169
%P 2195-2207

Markdown (Informal)

[Specializing Pre-trained Language Models for Better Relational Reasoning via Network Pruning](https://aclanthology.org/2022.findings-naacl.169) (Ren & Zhu, Findings 2022)

• Specializing Pre-trained Language Models for Better Relational Reasoning via Network Pruning (Ren & Zhu, Findings 2022)

ACL

• Siyu Ren and Kenny Zhu. 2022. Specializing Pre-trained Language Models for Better Relational Reasoning via Network Pruning. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 2195–2207, Seattle, United States. Association for Computational Linguistics.