<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="ostheimer-etal-2023-call">
    <titleInfo>
        <title>A Call for Standardization and Validation of Text Style Transfer Evaluation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Phil</namePart>
        <namePart type="family">Ostheimer</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mayank</namePart>
        <namePart type="given">Kumar</namePart>
        <namePart type="family">Nagda</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Marius</namePart>
        <namePart type="family">Kloft</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sophie</namePart>
        <namePart type="family">Fellenz</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jordan</namePart>
            <namePart type="family">Boyd-Graber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Naoaki</namePart>
            <namePart type="family">Okazaki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Text Style Transfer (TST) evaluation is, in practice, inconsistent. Therefore, we conduct a meta-analysis on human and automated TST evaluation and experimentation that thoroughly examines existing literature in the field. The meta-analysis reveals a substantial standardization gap in human and automated evaluation. In addition, we also find a validation gap: only few automated metrics have been validated using human experiments. To this end, we thoroughly scrutinize both the standardization and validation gap and reveal the resulting pitfalls. This work also paves the way to close the standardization and validation gap in TST evaluation by calling out requirements to be met by future research.</abstract>
    <identifier type="citekey">ostheimer-etal-2023-call</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-acl.687</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-acl.687</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>10791</start>
            <end>10815</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T A Call for Standardization and Validation of Text Style Transfer Evaluation
%A Ostheimer, Phil
%A Nagda, Mayank Kumar
%A Kloft, Marius
%A Fellenz, Sophie
%Y Rogers, Anna
%Y Boyd-Graber, Jordan
%Y Okazaki, Naoaki
%S Findings of the Association for Computational Linguistics: ACL 2023
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F ostheimer-etal-2023-call
%X Text Style Transfer (TST) evaluation is, in practice, inconsistent. Therefore, we conduct a meta-analysis on human and automated TST evaluation and experimentation that thoroughly examines existing literature in the field. The meta-analysis reveals a substantial standardization gap in human and automated evaluation. In addition, we also find a validation gap: only few automated metrics have been validated using human experiments. To this end, we thoroughly scrutinize both the standardization and validation gap and reveal the resulting pitfalls. This work also paves the way to close the standardization and validation gap in TST evaluation by calling out requirements to be met by future research.
%R 10.18653/v1/2023.findings-acl.687
%U https://aclanthology.org/2023.findings-acl.687
%U https://doi.org/10.18653/v1/2023.findings-acl.687
%P 10791-10815

Markdown (Informal)

[A Call for Standardization and Validation of Text Style Transfer Evaluation](https://aclanthology.org/2023.findings-acl.687) (Ostheimer et al., Findings 2023)

• A Call for Standardization and Validation of Text Style Transfer Evaluation (Ostheimer et al., Findings 2023)

ACL

• Phil Ostheimer, Mayank Kumar Nagda, Marius Kloft, and Sophie Fellenz. 2023. A Call for Standardization and Validation of Text Style Transfer Evaluation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 10791–10815, Toronto, Canada. Association for Computational Linguistics.