<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="erofeev-etal-2021-cushlepor">
    <titleInfo>
        <title>cushLEPOR uses LABSE distilled knowledge to improve correlation with human translation evaluations</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Gleb</namePart>
        <namePart type="family">Erofeev</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Irina</namePart>
        <namePart type="family">Sorokina</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lifeng</namePart>
        <namePart type="family">Han</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Serge</namePart>
        <namePart type="family">Gladkoff</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of Machine Translation Summit XVIII: Users and Providers Track</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Janice</namePart>
            <namePart type="family">Campbell</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ben</namePart>
            <namePart type="family">Huyck</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Stephen</namePart>
            <namePart type="family">Larocca</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jay</namePart>
            <namePart type="family">Marciano</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Konstantin</namePart>
            <namePart type="family">Savenkov</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Alex</namePart>
            <namePart type="family">Yanishevsky</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Machine Translation in the Americas</publisher>
            <place>
                <placeTerm type="text">Virtual</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Automatic MT evaluation metrics are indispensable for MT research. Augmented metrics such as hLEPOR include broader evaluation factors (recall and position difference penalty) in addition to the factors used in BLEU (sentence length, precision), and demonstrated higher accuracy. However, the obstacles preventing the wide use of hLEPOR were the lack of easy portable Python package and empirical weighting parameters that were tuned by manual work. This project addresses the above issues by offering a Python implementation of hLEPOR and automatic tuning of the parameters. We use existing translation memories (TM) as reference set and distillation modeling with LaBSE (Language-Agnostic BERT Sentence Embedding) to calibrate parameters for custom hLEPOR (cushLEPOR). cushLEPOR maximizes the correlation between hLEPOR and the distilling model similarity score towards reference. It can be used quickly and precisely to evaluate MT output from different engines, without need of manual weight tuning for optimization. In this session you will learn how to tune hLEPOR to obtain automatic custom-tuned cushLEPOR metric far more precise than BLEU. The method does not require costly human evaluations, existing TM is taken as a reference translation set, and cushLEPOR is created to select the best MT engine for the reference data-set.</abstract>
    <identifier type="citekey">erofeev-etal-2021-cushlepor</identifier>
    <location>
        <url>https://aclanthology.org/2021.mtsummit-up.28</url>
    </location>
    <part>
        <date>2021-08</date>
        <extent unit="page">
            <start>421</start>
            <end>439</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T cushLEPOR uses LABSE distilled knowledge to improve correlation with human translation evaluations
%A Erofeev, Gleb
%A Sorokina, Irina
%A Han, Lifeng
%A Gladkoff, Serge
%Y Campbell, Janice
%Y Huyck, Ben
%Y Larocca, Stephen
%Y Marciano, Jay
%Y Savenkov, Konstantin
%Y Yanishevsky, Alex
%S Proceedings of Machine Translation Summit XVIII: Users and Providers Track
%D 2021
%8 August
%I Association for Machine Translation in the Americas
%C Virtual
%F erofeev-etal-2021-cushlepor
%X Automatic MT evaluation metrics are indispensable for MT research. Augmented metrics such as hLEPOR include broader evaluation factors (recall and position difference penalty) in addition to the factors used in BLEU (sentence length, precision), and demonstrated higher accuracy. However, the obstacles preventing the wide use of hLEPOR were the lack of easy portable Python package and empirical weighting parameters that were tuned by manual work. This project addresses the above issues by offering a Python implementation of hLEPOR and automatic tuning of the parameters. We use existing translation memories (TM) as reference set and distillation modeling with LaBSE (Language-Agnostic BERT Sentence Embedding) to calibrate parameters for custom hLEPOR (cushLEPOR). cushLEPOR maximizes the correlation between hLEPOR and the distilling model similarity score towards reference. It can be used quickly and precisely to evaluate MT output from different engines, without need of manual weight tuning for optimization. In this session you will learn how to tune hLEPOR to obtain automatic custom-tuned cushLEPOR metric far more precise than BLEU. The method does not require costly human evaluations, existing TM is taken as a reference translation set, and cushLEPOR is created to select the best MT engine for the reference data-set.
%U https://aclanthology.org/2021.mtsummit-up.28
%P 421-439

Markdown (Informal)

[cushLEPOR uses LABSE distilled knowledge to improve correlation with human translation evaluations](https://aclanthology.org/2021.mtsummit-up.28) (Erofeev et al., MTSummit 2021)

• cushLEPOR uses LABSE distilled knowledge to improve correlation with human translation evaluations (Erofeev et al., MTSummit 2021)

ACL

• Gleb Erofeev, Irina Sorokina, Lifeng Han, and Serge Gladkoff. 2021. cushLEPOR uses LABSE distilled knowledge to improve correlation with human translation evaluations. In Proceedings of Machine Translation Summit XVIII: Users and Providers Track, pages 421–439, Virtual. Association for Machine Translation in the Americas.