<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="anderson-gomez-rodriguez-2020-distilling">
    <titleInfo>
        <title>Distilling Neural Networks for Greener and Faster Dependency Parsing</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Mark</namePart>
        <namePart type="family">Anderson</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Carlos</namePart>
        <namePart type="family">Gómez-Rodríguez</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The carbon footprint of natural language processing research has been increasing in recent years due to its reliance on large and inefficient neural network implementations. Distillation is a network compression technique which attempts to impart knowledge from a large model to a smaller one. We use teacher-student distillation to improve the efficiency of the Biaffine dependency parser which obtains state-of-the-art performance with respect to accuracy and parsing speed (Dozat and Manning, 2017). When distilling to 20% of the original model’s trainable parameters, we only observe an average decrease of ∼1 point for both UAS and LAS across a number of diverse Universal Dependency treebanks while being 2.30x (1.19x) faster than the baseline model on CPU (GPU) at inference time. We also observe a small increase in performance when compressing to 80% for some treebanks. Finally, through distillation we attain a parser which is not only faster but also more accurate than the fastest modern parser on the Penn Treebank.</abstract>
    <identifier type="citekey">anderson-gomez-rodriguez-2020-distilling</identifier>
    <identifier type="doi">10.18653/v1/2020.iwpt-1.2</identifier>
    <location>
        <url>https://aclanthology.org/2020.iwpt-1.2</url>
    </location>
    <part>
        <date>2020-07</date>
        <extent unit="page">
            <start>2</start>
            <end>13</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Distilling Neural Networks for Greener and Faster Dependency Parsing
%A Anderson, Mark
%A Gómez-Rodríguez, Carlos
%S Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies
%D 2020
%8 July
%I Association for Computational Linguistics
%C Online
%F anderson-gomez-rodriguez-2020-distilling
%X The carbon footprint of natural language processing research has been increasing in recent years due to its reliance on large and inefficient neural network implementations. Distillation is a network compression technique which attempts to impart knowledge from a large model to a smaller one. We use teacher-student distillation to improve the efficiency of the Biaffine dependency parser which obtains state-of-the-art performance with respect to accuracy and parsing speed (Dozat and Manning, 2017). When distilling to 20% of the original model’s trainable parameters, we only observe an average decrease of ∼1 point for both UAS and LAS across a number of diverse Universal Dependency treebanks while being 2.30x (1.19x) faster than the baseline model on CPU (GPU) at inference time. We also observe a small increase in performance when compressing to 80% for some treebanks. Finally, through distillation we attain a parser which is not only faster but also more accurate than the fastest modern parser on the Penn Treebank.
%R 10.18653/v1/2020.iwpt-1.2
%U https://aclanthology.org/2020.iwpt-1.2
%U https://doi.org/10.18653/v1/2020.iwpt-1.2
%P 2-13

Markdown (Informal)

[Distilling Neural Networks for Greener and Faster Dependency Parsing](https://aclanthology.org/2020.iwpt-1.2) (Anderson & Gómez-Rodríguez, IWPT 2020)

• Distilling Neural Networks for Greener and Faster Dependency Parsing (Anderson & Gómez-Rodríguez, IWPT 2020)

ACL

• Mark Anderson and Carlos Gómez-Rodríguez. 2020. Distilling Neural Networks for Greener and Faster Dependency Parsing. In Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies, pages 2–13, Online. Association for Computational Linguistics.