<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="arvan-etal-2022-reproducibility-computational">
    <titleInfo>
        <title>Reproducibility in Computational Linguistics: Is Source Code Enough?</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Mohammad</namePart>
        <namePart type="family">Arvan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Luís</namePart>
        <namePart type="family">Pina</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Natalie</namePart>
        <namePart type="family">Parde</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yoav</namePart>
            <namePart type="family">Goldberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zornitsa</namePart>
            <namePart type="family">Kozareva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The availability of source code has been put forward as one of the most critical factors for improving the reproducibility of scientific research. This work studies trends in source code availability at major computational linguistics conferences, namely, ACL, EMNLP, LREC, NAACL, and COLING. We observe positive trends, especially in conferences that actively promote reproducibility. We follow this by conducting a reproducibility study of eight papers published in EMNLP 2021, finding that source code releases leave much to be desired. Moving forward, we suggest all conferences require self-contained artifacts and provide a venue to evaluate such artifacts at the time of publication. Authors can include small-scale experiments and explicit scripts to generate each result to improve the reproducibility of their work.</abstract>
    <identifier type="citekey">arvan-etal-2022-reproducibility-computational</identifier>
    <identifier type="doi">10.18653/v1/2022.emnlp-main.150</identifier>
    <location>
        <url>https://aclanthology.org/2022.emnlp-main.150/</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>2350</start>
            <end>2361</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Reproducibility in Computational Linguistics: Is Source Code Enough?
%A Arvan, Mohammad
%A Pina, Luís
%A Parde, Natalie
%Y Goldberg, Yoav
%Y Kozareva, Zornitsa
%Y Zhang, Yue
%S Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates
%F arvan-etal-2022-reproducibility-computational
%X The availability of source code has been put forward as one of the most critical factors for improving the reproducibility of scientific research. This work studies trends in source code availability at major computational linguistics conferences, namely, ACL, EMNLP, LREC, NAACL, and COLING. We observe positive trends, especially in conferences that actively promote reproducibility. We follow this by conducting a reproducibility study of eight papers published in EMNLP 2021, finding that source code releases leave much to be desired. Moving forward, we suggest all conferences require self-contained artifacts and provide a venue to evaluate such artifacts at the time of publication. Authors can include small-scale experiments and explicit scripts to generate each result to improve the reproducibility of their work.
%R 10.18653/v1/2022.emnlp-main.150
%U https://aclanthology.org/2022.emnlp-main.150/
%U https://doi.org/10.18653/v1/2022.emnlp-main.150
%P 2350-2361

Markdown (Informal)

[Reproducibility in Computational Linguistics: Is Source Code Enough?](https://aclanthology.org/2022.emnlp-main.150/) (Arvan et al., EMNLP 2022)

• Reproducibility in Computational Linguistics: Is Source Code Enough? (Arvan et al., EMNLP 2022)

ACL

• Mohammad Arvan, Luís Pina, and Natalie Parde. 2022. Reproducibility in Computational Linguistics: Is Source Code Enough?. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2350–2361, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.