<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="singh-etal-2026-script">
    <titleInfo>
        <title>Script Matters: Benchmarking Open-Source Machine Translation for Manipuri in Meetei-Mayek and Bengali Script</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Kshetrimayum</namePart>
        <namePart type="given">Boynao</namePart>
        <namePart type="family">Singh</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Asif</namePart>
        <namePart type="family">Ekbal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Partha</namePart>
        <namePart type="family">Pakray</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Eleventh Conference on Machine Translation</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Barry</namePart>
            <namePart type="family">Haddow</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tom</namePart>
            <namePart type="family">Kocmi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Philipp</namePart>
            <namePart type="family">Koehn</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Christof</namePart>
            <namePart type="family">Monz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-474-3</identifier>
    </relatedItem>
    <abstract>Manipuri (Meiteilon) officially recognised the Meetei-Mayek script alongside the Bengali script in 2021, with both scripts permitted for concurrent use during a 10-year transition period. However, the FLORES-200 devtest release still encodes Manipuri only as Bengali script, leaving the language’s current official script without a standard benchmark. We address this gap by contributing the Meetei-Mayek script (mni_Mtei) layer, aligned with the existing Bengali-script (mni_Beng) data, to form a four-way parallel resource comprising English, Hindi, Meetei-Mayek Manipuri, and Bengali-script Manipuri, with 1,012 sentences per language. This resource enables separate evaluation of open-source MT and LLM systems on the two Manipuri scripts, rather than treating them as a single “Manipuri” target. Across three IndicTrans2 checkpoint families, Sarvam-Translate, and five general-purpose LLMs, evaluated using six automatic metrics, we find that legacy Bengali-script output often scores higher than Meetei-Mayek despite the latter being the official script; IndicTrans2’s family degrades sharply on Bengali-script Manipuri; Sarvam-Translate cannot produce Bengali script at all; and none of the five general-purpose LLMs we test can reliably read or write Meetei-Mayek. Script identity is therefore a first-class evaluation variable for Manipuri MT: a single “Manipuri” score can mask large, direction-dependent quality gaps.</abstract>
    <identifier type="citekey">singh-etal-2026-script</identifier>
    <location>
        <url>https://aclanthology.org/2026.wmt-main.200/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>2561</start>
            <end>2570</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Script Matters: Benchmarking Open-Source Machine Translation for Manipuri in Meetei-Mayek and Bengali Script
%A Singh, Kshetrimayum Boynao
%A Ekbal, Asif
%A Pakray, Partha
%Y Haddow, Barry
%Y Kocmi, Tom
%Y Koehn, Philipp
%Y Monz, Christof
%S Proceedings of the Eleventh Conference on Machine Translation
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-474-3
%F singh-etal-2026-script
%X Manipuri (Meiteilon) officially recognised the Meetei-Mayek script alongside the Bengali script in 2021, with both scripts permitted for concurrent use during a 10-year transition period. However, the FLORES-200 devtest release still encodes Manipuri only as Bengali script, leaving the language’s current official script without a standard benchmark. We address this gap by contributing the Meetei-Mayek script (mni_Mtei) layer, aligned with the existing Bengali-script (mni_Beng) data, to form a four-way parallel resource comprising English, Hindi, Meetei-Mayek Manipuri, and Bengali-script Manipuri, with 1,012 sentences per language. This resource enables separate evaluation of open-source MT and LLM systems on the two Manipuri scripts, rather than treating them as a single “Manipuri” target. Across three IndicTrans2 checkpoint families, Sarvam-Translate, and five general-purpose LLMs, evaluated using six automatic metrics, we find that legacy Bengali-script output often scores higher than Meetei-Mayek despite the latter being the official script; IndicTrans2’s family degrades sharply on Bengali-script Manipuri; Sarvam-Translate cannot produce Bengali script at all; and none of the five general-purpose LLMs we test can reliably read or write Meetei-Mayek. Script identity is therefore a first-class evaluation variable for Manipuri MT: a single “Manipuri” score can mask large, direction-dependent quality gaps.
%U https://aclanthology.org/2026.wmt-main.200/
%P 2561-2570

Markdown (Informal)

[Script Matters: Benchmarking Open-Source Machine Translation for Manipuri in Meetei-Mayek and Bengali Script](https://aclanthology.org/2026.wmt-main.200/) (Singh et al., WMT 2026)

• Script Matters: Benchmarking Open-Source Machine Translation for Manipuri in Meetei-Mayek and Bengali Script (Singh et al., WMT 2026)

ACL

• Kshetrimayum Boynao Singh, Asif Ekbal, and Partha Pakray. 2026. Script Matters: Benchmarking Open-Source Machine Translation for Manipuri in Meetei-Mayek and Bengali Script. In Proceedings of the Eleventh Conference on Machine Translation, pages 2561–2570, Budapest, Hungary. Association for Computational Linguistics.