Scoring the Translation: On Target Automatic Keyword-Based Evaluation of Machine Translation in the Sports Domain

Steinþór Steingrímsson; Einar Freyr Sigurðsson

doi:10.63317/3ctpkpj58fwf

Scoring the Translation: On Target Automatic Keyword-Based Evaluation of Machine Translation in the Sports Domain

Steinþór Steingrímsson, Einar Freyr Sigurðsson

Correct Metadata for

Use this form to create a GitHub issue with structured data describing the correction. You will need a GitHub account. Once you create that issue, the correction will be reviewed by a staff member.

⚠️ Mobile Users: Submitting this form to create a new issue will only work with github.com, not the GitHub Mobile app.

Important: The Anthology treat PDFs as authoritative. Please use this form only to correct data that is out of line with the PDF. See our corrections guidelines if you need to change the PDF.

Title Adjust the title. Retain tags such as <fixed-case>.

Authors Adjust author names and order to match the PDF.

Abstract Correct abstract if needed. Retain XML formatting tags such as <tex-math>. You may use ... for bold, ... for italic, ... for underline, <sc>...</sc> for small-caps, <tt>...<tt> for typewriter text, <url>...</url> for URLs, <a href=...> for hyperlinks, and <par/> for paragraph breaks.

Verification against PDF Ensure that the new title/authors match the snapshot below. (If there is no snapshot or it is too small, consult the external publication .)

Authors concatenated from the text boxes above:

ALL author names match the snapshot above—including middle initials, hyphens, and accents.

Abstract

We take a closer look at the results of a recent translation shared task at WMT 2025 (the Conference on Machine Translation) and analyse the errors in the output of the four highest-scoring systems. We revise the automatic evaluation method used in Sigurðsson et al. (2025) and compare it to manual evaluation of six machine translation systems. We find that our results are in line with the manual evaluation, indicating that the test suite can be well suited for evaluating machine translation in this domain. Finally, we publish a list of domain-specific sports terms, namely, in the domains of basketball, chess, football, golf and gymnastics.

Anthology ID:: 2026.lrec-1.697
Volume:: Proceedings of the Fifteenth Language Resources and Evaluation Conference
Month:: May
Year:: 2026
Address:: Palma de Mallorca, Spain
Editors:: Stelios Piperidis, Núria Bel, Henk van den Heuvel, Nancy Ide, Simon Krek, Antonio Toral
Venue:: LREC
SIG:
Publisher:: ELRA Language Resource Association
Note:
Pages:: 8862–8871
Language:
External URL:: https://lrec.elra.info/lrec2026-main-697
DOI:: 10.63317/3ctpkpj58fwf
Bibkey:
Cite (ACL):: Steinþór Steingrímsson and Einar Freyr Sigurðsson. 2026. Scoring the Translation: On Target Automatic Keyword-Based Evaluation of Machine Translation in the Sports Domain. In Proceedings of the Fifteenth Language Resources and Evaluation Conference, pages 8862–8871, Palma de Mallorca, Spain. ELRA Language Resource Association.
Cite (Informal):: Scoring the Translation: On Target Automatic Keyword-Based Evaluation of Machine Translation in the Sports Domain (Steingrímsson & Sigurðsson, LREC 2026)
Copy Citation:

External Cite Search Fix data

Export citation

BibTeX
MODS XML
Endnote
Preformatted

@inproceedings{steingrimsson-sigurdsson-2026-scoring,
    title = "Scoring the Translation: On Target Automatic Keyword-Based Evaluation of Machine Translation in the Sports Domain",
    author = "Steingr{\'i}msson, Stein{\th}{\'o}r  and
      Sigur{\dh}sson, Einar Freyr",
    editor = "Piperidis, Stelios  and
      Bel, N{\'u}ria  and
      van den Heuvel, Henk  and
      Ide, Nancy  and
      Krek, Simon  and
      Toral, Antonio",
    booktitle = "Proceedings of the Fifteenth Language Resources and Evaluation Conference",
    month = may,
    year = "2026",
    address = "Palma de Mallorca, Spain",
    publisher = "ELRA Language Resource Association",
    url = "https://aclanthology.org/2026.lrec-1.697/",
    doi = "10.63317/3ctpkpj58fwf",
    pages = "8862--8871",
    abstract = "We take a closer look at the results of a recent translation shared task at WMT 2025 (the Conference on Machine Translation) and analyse the errors in the output of the four highest-scoring systems. We revise the automatic evaluation method used in Sigur{\dh}sson et al. (2025) and compare it to manual evaluation of six machine translation systems. We find that our results are in line with the manual evaluation, indicating that the test suite can be well suited for evaluating machine translation in this domain. Finally, we publish a list of domain-specific sports terms, namely, in the domains of basketball, chess, football, golf and gymnastics."
}

Download as File

<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="steingrimsson-sigurdsson-2026-scoring">
    <titleInfo>
        <title>Scoring the Translation: On Target Automatic Keyword-Based Evaluation of Machine Translation in the Sports Domain</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Stein\thór</namePart>
        <namePart type="family">Steingrímsson</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Einar</namePart>
        <namePart type="given">Freyr</namePart>
        <namePart type="family">Sigur\dhsson</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Fifteenth Language Resources and Evaluation Conference</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Stelios</namePart>
            <namePart type="family">Piperidis</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Núria</namePart>
            <namePart type="family">Bel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Henk</namePart>
            <namePart type="family">van den Heuvel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nancy</namePart>
            <namePart type="family">Ide</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Simon</namePart>
            <namePart type="family">Krek</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Antonio</namePart>
            <namePart type="family">Toral</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA Language Resource Association</publisher>
            <place>
                <placeTerm type="text">Palma de Mallorca, Spain</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We take a closer look at the results of a recent translation shared task at WMT 2025 (the Conference on Machine Translation) and analyse the errors in the output of the four highest-scoring systems. We revise the automatic evaluation method used in Sigur\dhsson et al. (2025) and compare it to manual evaluation of six machine translation systems. We find that our results are in line with the manual evaluation, indicating that the test suite can be well suited for evaluating machine translation in this domain. Finally, we publish a list of domain-specific sports terms, namely, in the domains of basketball, chess, football, golf and gymnastics.</abstract>
    <identifier type="citekey">steingrimsson-sigurdsson-2026-scoring</identifier>
    <identifier type="doi">10.63317/3ctpkpj58fwf</identifier>
    <location>
        <url>https://aclanthology.org/2026.lrec-1.697/</url>
    </location>
    <part>
        <date>2026-05</date>
        <extent unit="page">
            <start>8862</start>
            <end>8871</end>
        </extent>
    </part>
</mods>
</modsCollection>

Download as File

%0 Conference Proceedings
%T Scoring the Translation: On Target Automatic Keyword-Based Evaluation of Machine Translation in the Sports Domain
%A Steingrímsson, Stein\thór
%A Sigur\dhsson, Einar Freyr
%Y Piperidis, Stelios
%Y Bel, Núria
%Y van den Heuvel, Henk
%Y Ide, Nancy
%Y Krek, Simon
%Y Toral, Antonio
%S Proceedings of the Fifteenth Language Resources and Evaluation Conference
%D 2026
%8 May
%I ELRA Language Resource Association
%C Palma de Mallorca, Spain
%F steingrimsson-sigurdsson-2026-scoring
%X We take a closer look at the results of a recent translation shared task at WMT 2025 (the Conference on Machine Translation) and analyse the errors in the output of the four highest-scoring systems. We revise the automatic evaluation method used in Sigur\dhsson et al. (2025) and compare it to manual evaluation of six machine translation systems. We find that our results are in line with the manual evaluation, indicating that the test suite can be well suited for evaluating machine translation in this domain. Finally, we publish a list of domain-specific sports terms, namely, in the domains of basketball, chess, football, golf and gymnastics.
%R 10.63317/3ctpkpj58fwf
%U https://aclanthology.org/2026.lrec-1.697/
%U https://doi.org/10.63317/3ctpkpj58fwf
%P 8862-8871

Download as File

Markdown (Informal)

[Scoring the Translation: On Target Automatic Keyword-Based Evaluation of Machine Translation in the Sports Domain](https://aclanthology.org/2026.lrec-1.697/) (Steingrímsson & Sigurðsson, LREC 2026)

Scoring the Translation: On Target Automatic Keyword-Based Evaluation of Machine Translation in the Sports Domain (Steingrímsson & Sigurðsson, LREC 2026)

ACL

Steinþór Steingrímsson and Einar Freyr Sigurðsson. 2026. Scoring the Translation: On Target Automatic Keyword-Based Evaluation of Machine Translation in the Sports Domain. In Proceedings of the Fifteenth Language Resources and Evaluation Conference, pages 8862–8871, Palma de Mallorca, Spain. ELRA Language Resource Association.