<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="belz-etal-2022-2022">
    <titleInfo>
        <title>The 2022 ReproGen Shared Task on Reproducibility of Evaluations in NLG: Overview and Results</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Anya</namePart>
        <namePart type="family">Belz</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Anastasia</namePart>
        <namePart type="family">Shimorina</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Maja</namePart>
        <namePart type="family">Popović</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ehud</namePart>
        <namePart type="family">Reiter</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 15th International Conference on Natural Language Generation: Generation Challenges</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Waterville, Maine, USA and virtual meeting</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Against a background of growing interest in reproducibility in NLP and ML, and as part of an ongoing research programme designed to develop theory and practice of reproducibility assessment in NLP, we organised the second shared task on reproducibility of evaluations in NLG, ReproGen 2022. This paper describes the shared task, summarises results from the reproduction studies submitted, and provides further comparative analysis of the results. Out of six initial team registrations, we received submissions from five teams. Meta-analysis of the five reproduction studies revealed varying degrees of reproducibility, and allowed further tentative conclusions about what types of eval- uation tend to have better reproducibility.</abstract>
    <identifier type="citekey">belz-etal-2022-2022</identifier>
    <location>
        <url>https://aclanthology.org/2022.inlg-genchal.8</url>
    </location>
    <part>
        <date>2022-07</date>
        <extent unit="page">
            <start>43</start>
            <end>51</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T The 2022 ReproGen Shared Task on Reproducibility of Evaluations in NLG: Overview and Results
%A Belz, Anya
%A Shimorina, Anastasia
%A Popović, Maja
%A Reiter, Ehud
%S Proceedings of the 15th International Conference on Natural Language Generation: Generation Challenges
%D 2022
%8 July
%I Association for Computational Linguistics
%C Waterville, Maine, USA and virtual meeting
%F belz-etal-2022-2022
%X Against a background of growing interest in reproducibility in NLP and ML, and as part of an ongoing research programme designed to develop theory and practice of reproducibility assessment in NLP, we organised the second shared task on reproducibility of evaluations in NLG, ReproGen 2022. This paper describes the shared task, summarises results from the reproduction studies submitted, and provides further comparative analysis of the results. Out of six initial team registrations, we received submissions from five teams. Meta-analysis of the five reproduction studies revealed varying degrees of reproducibility, and allowed further tentative conclusions about what types of eval- uation tend to have better reproducibility.
%U https://aclanthology.org/2022.inlg-genchal.8
%P 43-51

Markdown (Informal)

[The 2022 ReproGen Shared Task on Reproducibility of Evaluations in NLG: Overview and Results](https://aclanthology.org/2022.inlg-genchal.8) (Belz et al., INLG 2022)

• The 2022 ReproGen Shared Task on Reproducibility of Evaluations in NLG: Overview and Results (Belz et al., INLG 2022)

ACL

• Anya Belz, Anastasia Shimorina, Maja Popović, and Ehud Reiter. 2022. The 2022 ReproGen Shared Task on Reproducibility of Evaluations in NLG: Overview and Results. In Proceedings of the 15th International Conference on Natural Language Generation: Generation Challenges, pages 43–51, Waterville, Maine, USA and virtual meeting. Association for Computational Linguistics.