<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="jon-etal-2026-current">
    <titleInfo>
        <title>Current state of LLMs for Arabic dialectal machine translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Josef</namePart>
        <namePart type="family">Jon</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Rawan</namePart>
        <namePart type="family">Bondok</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ondřej</namePart>
        <namePart type="family">Bojar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-03</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Rabat, Morocco</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>This work presents an evaluation of large language models (LLMs) for English to dialectal Arabic machine translation on the MADAR dataset. We evaluate both translation directions (English to Arabic and vice-versa) on 16 Arabic dialects. Our experiments cover a diverse set of models, including specialized Arabic models (Jais, Nile), multilingual models (Gemma, Command-R, Mistral, Aya), and commercial APIs (GPT-4.1). We employ multiple evaluation metrics: BLEU, CHRF, COMET (both reference-based and reference-less variants) and GEMBA (LLM-as-a-judge), as well as a small-scale manual evaluation, to assess translation quality. We discuss the challenges of automatic MT evaluation, especially in the context of Arabic dialects. We also evaluate the ability of LLMs to classify the dialect used in a text. The study offers insights into the capabilities and limitations of current LLMs for dialectal Arabic machine translation, particularly highlighting the difficulty of handling dialectal diversity, although the results may be influenced by possible training data contamination, which is always a concern with LLMs.</abstract>
    <identifier type="citekey">jon-etal-2026-current</identifier>
    <location>
        <url>https://aclanthology.org/2026.abjadnlp-1.41/</url>
    </location>
    <part>
        <date>2026-03</date>
        <extent unit="page">
            <start>329</start>
            <end>363</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Current state of LLMs for Arabic dialectal machine translation
%A Jon, Josef
%A Bondok, Rawan
%A Bojar, Ondřej
%S Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script
%D 2026
%8 March
%I Association for Computational Linguistics
%C Rabat, Morocco
%F jon-etal-2026-current
%X This work presents an evaluation of large language models (LLMs) for English to dialectal Arabic machine translation on the MADAR dataset. We evaluate both translation directions (English to Arabic and vice-versa) on 16 Arabic dialects. Our experiments cover a diverse set of models, including specialized Arabic models (Jais, Nile), multilingual models (Gemma, Command-R, Mistral, Aya), and commercial APIs (GPT-4.1). We employ multiple evaluation metrics: BLEU, CHRF, COMET (both reference-based and reference-less variants) and GEMBA (LLM-as-a-judge), as well as a small-scale manual evaluation, to assess translation quality. We discuss the challenges of automatic MT evaluation, especially in the context of Arabic dialects. We also evaluate the ability of LLMs to classify the dialect used in a text. The study offers insights into the capabilities and limitations of current LLMs for dialectal Arabic machine translation, particularly highlighting the difficulty of handling dialectal diversity, although the results may be influenced by possible training data contamination, which is always a concern with LLMs.
%U https://aclanthology.org/2026.abjadnlp-1.41/
%P 329-363

Markdown (Informal)

[Current state of LLMs for Arabic dialectal machine translation](https://aclanthology.org/2026.abjadnlp-1.41/) (Jon et al., AbjadNLP 2026)

• Current state of LLMs for Arabic dialectal machine translation (Jon et al., AbjadNLP 2026)

ACL

• Josef Jon, Rawan Bondok, and Ondřej Bojar. 2026. Current state of LLMs for Arabic dialectal machine translation. In Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script, pages 329–363, Rabat, Morocco. Association for Computational Linguistics.