<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="hazem-kageura-2026-coterm">
    <titleInfo>
        <title>CoTERM: A Consistency-Oriented Term Metric for MT System Evaluation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Amir</namePart>
        <namePart type="family">Hazem</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kyo</namePart>
        <namePart type="family">Kageura</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Fifteenth Language Resources and Evaluation Conference</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Stelios</namePart>
            <namePart type="family">Piperidis</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Núria</namePart>
            <namePart type="family">Bel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Henk</namePart>
            <namePart type="family">van den Heuvel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nancy</namePart>
            <namePart type="family">Ide</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Simon</namePart>
            <namePart type="family">Krek</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Antonio</namePart>
            <namePart type="family">Toral</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA Language Resource Association</publisher>
            <place>
                <placeTerm type="text">Palma de Mallorca, Spain</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Proper treatment of terms is an important and critical aspect in machine translation. It is therefore necessary to use appropriate metrics to evaluate MT system outputs from terminology perspective. However, despite the great improvements witnessed in the recent NMT and LLM models, MT system evaluation metrics that shed light on specific aspects of term translations are yet to be fully explored. In this paper, we propose CoTERM, a new metric for automatic evaluation of term translations based on the Herfindahl-Hirshman Index (HHI). CoTERM measures target term closeness to one or more reference translations, taking into account the fundamental criteria for translating terms, i.e. (i) accuracy; (ii) consistency at document or corpus levels; and (iii) appropriateness to the domain conventions with regard to term variations. The proposed metric correlates strongly with human raters, and empirical evaluations of a wide range of NMTs and LLMs show that the best MT systems in standard metrics are not necessarily the best at treating terms. CoTERM is thus shown to be highly useful for diagnosing MT systems’ term translation performance and conveniently seen as complementary to generic measures for MT system evaluations.</abstract>
    <identifier type="citekey">hazem-kageura-2026-coterm</identifier>
    <identifier type="doi">10.63317/3pc9e4hsupuk</identifier>
    <location>
        <url>https://aclanthology.org/2026.lrec-1.682/</url>
    </location>
    <part>
        <date>2026-05</date>
        <extent unit="page">
            <start>8639</start>
            <end>8661</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T CoTERM: A Consistency-Oriented Term Metric for MT System Evaluation
%A Hazem, Amir
%A Kageura, Kyo
%Y Piperidis, Stelios
%Y Bel, Núria
%Y van den Heuvel, Henk
%Y Ide, Nancy
%Y Krek, Simon
%Y Toral, Antonio
%S Proceedings of the Fifteenth Language Resources and Evaluation Conference
%D 2026
%8 May
%I ELRA Language Resource Association
%C Palma de Mallorca, Spain
%F hazem-kageura-2026-coterm
%X Proper treatment of terms is an important and critical aspect in machine translation. It is therefore necessary to use appropriate metrics to evaluate MT system outputs from terminology perspective. However, despite the great improvements witnessed in the recent NMT and LLM models, MT system evaluation metrics that shed light on specific aspects of term translations are yet to be fully explored. In this paper, we propose CoTERM, a new metric for automatic evaluation of term translations based on the Herfindahl-Hirshman Index (HHI). CoTERM measures target term closeness to one or more reference translations, taking into account the fundamental criteria for translating terms, i.e. (i) accuracy; (ii) consistency at document or corpus levels; and (iii) appropriateness to the domain conventions with regard to term variations. The proposed metric correlates strongly with human raters, and empirical evaluations of a wide range of NMTs and LLMs show that the best MT systems in standard metrics are not necessarily the best at treating terms. CoTERM is thus shown to be highly useful for diagnosing MT systems’ term translation performance and conveniently seen as complementary to generic measures for MT system evaluations.
%R 10.63317/3pc9e4hsupuk
%U https://aclanthology.org/2026.lrec-1.682/
%U https://doi.org/10.63317/3pc9e4hsupuk
%P 8639-8661

Markdown (Informal)

[CoTERM: A Consistency-Oriented Term Metric for MT System Evaluation](https://aclanthology.org/2026.lrec-1.682/) (Hazem & Kageura, LREC 2026)

• CoTERM: A Consistency-Oriented Term Metric for MT System Evaluation (Hazem & Kageura, LREC 2026)

ACL

• Amir Hazem and Kyo Kageura. 2026. CoTERM: A Consistency-Oriented Term Metric for MT System Evaluation. In Proceedings of the Fifteenth Language Resources and Evaluation Conference, pages 8639–8661, Palma de Mallorca, Spain. ELRA Language Resource Association.