<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="farina-etal-2023-distillation">
    <titleInfo>
        <title>Distillation of encoder-decoder transformers for sequence labelling</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Marco</namePart>
        <namePart type="family">Farina</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Duccio</namePart>
        <namePart type="family">Pappadopulo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Anant</namePart>
        <namePart type="family">Gupta</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Leslie</namePart>
        <namePart type="family">Huang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ozan</namePart>
        <namePart type="family">Irsoy</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Thamar</namePart>
        <namePart type="family">Solorio</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Andreas</namePart>
            <namePart type="family">Vlachos</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Isabelle</namePart>
            <namePart type="family">Augenstein</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Dubrovnik, Croatia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Driven by encouraging results on a wide range of tasks, the field of NLP is experiencing an accelerated race to develop bigger language models. This race for bigger models has also underscored the need to continue the pursuit of practical distillation approaches that can leverage the knowledge acquired by these big models in a compute-efficient manner. Having this goal in mind, we build on recent work to propose a hallucination-free framework for sequence tagging that is especially suited for distillation. We show empirical results of new state-of-the-art performance across multiple sequence labelling datasets and validate the usefulness of this framework for distilling a large model in a few-shot learning scenario.</abstract>
    <identifier type="citekey">farina-etal-2023-distillation</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-eacl.192</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-eacl.192</url>
    </location>
    <part>
        <date>2023-05</date>
        <extent unit="page">
            <start>2539</start>
            <end>2549</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Distillation of encoder-decoder transformers for sequence labelling
%A Farina, Marco
%A Pappadopulo, Duccio
%A Gupta, Anant
%A Huang, Leslie
%A Irsoy, Ozan
%A Solorio, Thamar
%Y Vlachos, Andreas
%Y Augenstein, Isabelle
%S Findings of the Association for Computational Linguistics: EACL 2023
%D 2023
%8 May
%I Association for Computational Linguistics
%C Dubrovnik, Croatia
%F farina-etal-2023-distillation
%X Driven by encouraging results on a wide range of tasks, the field of NLP is experiencing an accelerated race to develop bigger language models. This race for bigger models has also underscored the need to continue the pursuit of practical distillation approaches that can leverage the knowledge acquired by these big models in a compute-efficient manner. Having this goal in mind, we build on recent work to propose a hallucination-free framework for sequence tagging that is especially suited for distillation. We show empirical results of new state-of-the-art performance across multiple sequence labelling datasets and validate the usefulness of this framework for distilling a large model in a few-shot learning scenario.
%R 10.18653/v1/2023.findings-eacl.192
%U https://aclanthology.org/2023.findings-eacl.192
%U https://doi.org/10.18653/v1/2023.findings-eacl.192
%P 2539-2549

Markdown (Informal)

[Distillation of encoder-decoder transformers for sequence labelling](https://aclanthology.org/2023.findings-eacl.192) (Farina et al., Findings 2023)

• Distillation of encoder-decoder transformers for sequence labelling (Farina et al., Findings 2023)

ACL

• Marco Farina, Duccio Pappadopulo, Anant Gupta, Leslie Huang, Ozan Irsoy, and Thamar Solorio. 2023. Distillation of encoder-decoder transformers for sequence labelling. In Findings of the Association for Computational Linguistics: EACL 2023, pages 2539–2549, Dubrovnik, Croatia. Association for Computational Linguistics.