<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="tarnavskyi-etal-2022-ensembling">
    <titleInfo>
        <title>Ensembling and Knowledge Distilling of Large Sequence Taggers for Grammatical Error Correction</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Maksym</namePart>
        <namePart type="family">Tarnavskyi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Artem</namePart>
        <namePart type="family">Chernodub</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kostiantyn</namePart>
        <namePart type="family">Omelianchuk</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Smaranda</namePart>
            <namePart type="family">Muresan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Preslav</namePart>
            <namePart type="family">Nakov</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Aline</namePart>
            <namePart type="family">Villavicencio</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Dublin, Ireland</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this paper, we investigate improvements to the GEC sequence tagging architecture with a focus on ensembling of recent cutting-edge Transformer-based encoders in Large configurations. We encourage ensembling models by majority votes on span-level edits because this approach is tolerant to the model architecture and vocabulary size. Our best ensemble achieves a new SOTA result with an F₀.5 score of 76.05 on BEA-2019 (test), even without pre-training on synthetic datasets. In addition, we perform knowledge distillation with a trained ensemble to generate new synthetic training datasets, “Troy-Blogs” and “Troy-1BW”. Our best single sequence tagging model that is pretrained on the generated Troy- datasets in combination with the publicly available synthetic PIE dataset achieves a near-SOTA result with an F₀.5 score of 73.21 on BEA-2019 (test). The code, datasets, and trained models are publicly available.</abstract>
    <identifier type="citekey">tarnavskyi-etal-2022-ensembling</identifier>
    <identifier type="doi">10.18653/v1/2022.acl-long.266</identifier>
    <location>
        <url>https://aclanthology.org/2022.acl-long.266</url>
    </location>
    <part>
        <date>2022-05</date>
        <extent unit="page">
            <start>3842</start>
            <end>3852</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Ensembling and Knowledge Distilling of Large Sequence Taggers for Grammatical Error Correction
%A Tarnavskyi, Maksym
%A Chernodub, Artem
%A Omelianchuk, Kostiantyn
%Y Muresan, Smaranda
%Y Nakov, Preslav
%Y Villavicencio, Aline
%S Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
%D 2022
%8 May
%I Association for Computational Linguistics
%C Dublin, Ireland
%F tarnavskyi-etal-2022-ensembling
%X In this paper, we investigate improvements to the GEC sequence tagging architecture with a focus on ensembling of recent cutting-edge Transformer-based encoders in Large configurations. We encourage ensembling models by majority votes on span-level edits because this approach is tolerant to the model architecture and vocabulary size. Our best ensemble achieves a new SOTA result with an F₀.5 score of 76.05 on BEA-2019 (test), even without pre-training on synthetic datasets. In addition, we perform knowledge distillation with a trained ensemble to generate new synthetic training datasets, “Troy-Blogs” and “Troy-1BW”. Our best single sequence tagging model that is pretrained on the generated Troy- datasets in combination with the publicly available synthetic PIE dataset achieves a near-SOTA result with an F₀.5 score of 73.21 on BEA-2019 (test). The code, datasets, and trained models are publicly available.
%R 10.18653/v1/2022.acl-long.266
%U https://aclanthology.org/2022.acl-long.266
%U https://doi.org/10.18653/v1/2022.acl-long.266
%P 3842-3852

Markdown (Informal)

[Ensembling and Knowledge Distilling of Large Sequence Taggers for Grammatical Error Correction](https://aclanthology.org/2022.acl-long.266) (Tarnavskyi et al., ACL 2022)

• Ensembling and Knowledge Distilling of Large Sequence Taggers for Grammatical Error Correction (Tarnavskyi et al., ACL 2022)

ACL

• Maksym Tarnavskyi, Artem Chernodub, and Kostiantyn Omelianchuk. 2022. Ensembling and Knowledge Distilling of Large Sequence Taggers for Grammatical Error Correction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3842–3852, Dublin, Ireland. Association for Computational Linguistics.