<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="arvan-parde-2023-human">
    <titleInfo>
        <title>Human Evaluation Reproduction Report for Data-to-text Generation with Macro Planning</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Mohammad</namePart>
        <namePart type="family">Arvan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Natalie</namePart>
        <namePart type="family">Parde</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 3rd Workshop on Human Evaluation of NLP Systems</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anya</namePart>
            <namePart type="family">Belz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Maja</namePart>
            <namePart type="family">Popović</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ehud</namePart>
            <namePart type="family">Reiter</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Craig</namePart>
            <namePart type="family">Thomson</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">João</namePart>
            <namePart type="family">Sedoc</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>INCOMA Ltd., Shoumen, Bulgaria</publisher>
            <place>
                <placeTerm type="text">Varna, Bulgaria</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>This paper presents a partial reproduction study of Data-to-text Generation with Macro Planning by Puduppully et al. (2021). This work was conducted as part of the ReproHum project, a multi-lab effort to reproduce the results of NLP papers incorporating human evaluations. We follow the same instructions provided by the authors and the ReproHum team to the best of our abilities. We collect preference ratings for the following evaluation criteria in order: conciseness, coherence, and grammaticality. Our results are highly correlated with the original experiment. Nonetheless, we believe the presented results are insufficent to conclude that the Macro system proposed and developed by the original paper is superior compared to other systems. We suspect combining our results with the three other reproductions of this paper through the ReproHum project will paint a clearer picture. Overall, we hope that our work is a step towards a more transparent and reproducible research landscape.</abstract>
    <identifier type="citekey">arvan-parde-2023-human</identifier>
    <location>
        <url>https://aclanthology.org/2023.humeval-1.8</url>
    </location>
    <part>
        <date>2023-09</date>
        <extent unit="page">
            <start>89</start>
            <end>96</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Human Evaluation Reproduction Report for Data-to-text Generation with Macro Planning
%A Arvan, Mohammad
%A Parde, Natalie
%Y Belz, Anya
%Y Popović, Maja
%Y Reiter, Ehud
%Y Thomson, Craig
%Y Sedoc, João
%S Proceedings of the 3rd Workshop on Human Evaluation of NLP Systems
%D 2023
%8 September
%I INCOMA Ltd., Shoumen, Bulgaria
%C Varna, Bulgaria
%F arvan-parde-2023-human
%X This paper presents a partial reproduction study of Data-to-text Generation with Macro Planning by Puduppully et al. (2021). This work was conducted as part of the ReproHum project, a multi-lab effort to reproduce the results of NLP papers incorporating human evaluations. We follow the same instructions provided by the authors and the ReproHum team to the best of our abilities. We collect preference ratings for the following evaluation criteria in order: conciseness, coherence, and grammaticality. Our results are highly correlated with the original experiment. Nonetheless, we believe the presented results are insufficent to conclude that the Macro system proposed and developed by the original paper is superior compared to other systems. We suspect combining our results with the three other reproductions of this paper through the ReproHum project will paint a clearer picture. Overall, we hope that our work is a step towards a more transparent and reproducible research landscape.
%U https://aclanthology.org/2023.humeval-1.8
%P 89-96

Markdown (Informal)

[Human Evaluation Reproduction Report for Data-to-text Generation with Macro Planning](https://aclanthology.org/2023.humeval-1.8) (Arvan & Parde, HumEval-WS 2023)

• Human Evaluation Reproduction Report for Data-to-text Generation with Macro Planning (Arvan & Parde, HumEval-WS 2023)

ACL

• Mohammad Arvan and Natalie Parde. 2023. Human Evaluation Reproduction Report for Data-to-text Generation with Macro Planning. In Proceedings of the 3rd Workshop on Human Evaluation of NLP Systems, pages 89–96, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria.