<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="knowles-etal-2024-mslc24">
    <titleInfo>
        <title>MSLC24: Further Challenges for Metrics on a Wide Landscape of Translation Quality</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Rebecca</namePart>
        <namePart type="family">Knowles</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Samuel</namePart>
        <namePart type="family">Larkin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chi-Kiu</namePart>
        <namePart type="family">Lo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Ninth Conference on Machine Translation</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Barry</namePart>
            <namePart type="family">Haddow</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tom</namePart>
            <namePart type="family">Kocmi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Philipp</namePart>
            <namePart type="family">Koehn</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Christof</namePart>
            <namePart type="family">Monz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this second edition of the Metric Score Landscape Challenge (MSLC), we examine how automatic metrics for machine translation perform on a wide variety of machine translation output, ranging from very low quality systems to the types of high-quality systems submitted to the General MT shared task at WMT. We also explore metric results on specific types of data, such as empty strings, wrong- or mixed-language text, and more. We raise several alarms about inconsistencies in metric scores, some of which can be resolved by increasingly explicit instructions for metric use, while others highlight technical flaws.</abstract>
    <identifier type="citekey">knowles-etal-2024-mslc24</identifier>
    <location>
        <url>https://aclanthology.org/2024.wmt-1.34</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>475</start>
            <end>491</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T MSLC24: Further Challenges for Metrics on a Wide Landscape of Translation Quality
%A Knowles, Rebecca
%A Larkin, Samuel
%A Lo, Chi-Kiu
%Y Haddow, Barry
%Y Kocmi, Tom
%Y Koehn, Philipp
%Y Monz, Christof
%S Proceedings of the Ninth Conference on Machine Translation
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F knowles-etal-2024-mslc24
%X In this second edition of the Metric Score Landscape Challenge (MSLC), we examine how automatic metrics for machine translation perform on a wide variety of machine translation output, ranging from very low quality systems to the types of high-quality systems submitted to the General MT shared task at WMT. We also explore metric results on specific types of data, such as empty strings, wrong- or mixed-language text, and more. We raise several alarms about inconsistencies in metric scores, some of which can be resolved by increasingly explicit instructions for metric use, while others highlight technical flaws.
%U https://aclanthology.org/2024.wmt-1.34
%P 475-491

Markdown (Informal)

[MSLC24: Further Challenges for Metrics on a Wide Landscape of Translation Quality](https://aclanthology.org/2024.wmt-1.34) (Knowles et al., WMT 2024)

• MSLC24: Further Challenges for Metrics on a Wide Landscape of Translation Quality (Knowles et al., WMT 2024)

ACL

• Rebecca Knowles, Samuel Larkin, and Chi-Kiu Lo. 2024. MSLC24: Further Challenges for Metrics on a Wide Landscape of Translation Quality. In Proceedings of the Ninth Conference on Machine Translation, pages 475–491, Miami, Florida, USA. Association for Computational Linguistics.