<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yakhni-chehab-2025-llms">
    <titleInfo>
        <title>Can LLMs Translate Cultural Nuance in Dialects? A Case Study on Lebanese Arabic</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Silvana</namePart>
        <namePart type="family">Yakhni</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ali</namePart>
        <namePart type="family">Chehab</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-01</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 1st Workshop on NLP for Languages Using Arabic Script</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Mo</namePart>
            <namePart type="family">El-Haj</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, UAE</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Machine Translation (MT) of Arabic-script languages presents unique challenges due to their vast linguistic diversity and lack of standardization. This paper focuses on the Lebanese dialect, investigating the effectiveness of Large Language Models (LLMs) in handling culturally-aware translations. We identify critical limitations in existing Lebanese-English parallel datasets, particularly their non-native nature and lack of cultural context. To address these gaps, we introduce a new culturally-rich dataset derived from the Language Wave (LW) podcast. We evaluate the performance of LLMs: Jais, AceGPT, Cohere, and GPT-4 models against Neural Machine Translation (NMT) systems: NLLB-200, and Google Translate. Our findings reveal that while both architectures perform similarly on non-native datasets, LLMs demonstrate superior capabilities in preserving cultural nuances when handling authentic Lebanese content. Additionally, we validate xCOMET as a reliable metric for evaluating the quality of Arabic dialect translation, showing a strong correlation with human judgment. This work contributes to the growing field of Culturally-Aware Machine Translation and highlights the importance of authentic, culturally representative datasets in advancing low-resource translation systems.</abstract>
    <identifier type="citekey">yakhni-chehab-2025-llms</identifier>
    <location>
        <url>https://aclanthology.org/2025.abjadnlp-1.13/</url>
    </location>
    <part>
        <date>2025-01</date>
        <extent unit="page">
            <start>114</start>
            <end>135</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Can LLMs Translate Cultural Nuance in Dialects? A Case Study on Lebanese Arabic
%A Yakhni, Silvana
%A Chehab, Ali
%Y El-Haj, Mo
%S Proceedings of the 1st Workshop on NLP for Languages Using Arabic Script
%D 2025
%8 January
%I Association for Computational Linguistics
%C Abu Dhabi, UAE
%F yakhni-chehab-2025-llms
%X Machine Translation (MT) of Arabic-script languages presents unique challenges due to their vast linguistic diversity and lack of standardization. This paper focuses on the Lebanese dialect, investigating the effectiveness of Large Language Models (LLMs) in handling culturally-aware translations. We identify critical limitations in existing Lebanese-English parallel datasets, particularly their non-native nature and lack of cultural context. To address these gaps, we introduce a new culturally-rich dataset derived from the Language Wave (LW) podcast. We evaluate the performance of LLMs: Jais, AceGPT, Cohere, and GPT-4 models against Neural Machine Translation (NMT) systems: NLLB-200, and Google Translate. Our findings reveal that while both architectures perform similarly on non-native datasets, LLMs demonstrate superior capabilities in preserving cultural nuances when handling authentic Lebanese content. Additionally, we validate xCOMET as a reliable metric for evaluating the quality of Arabic dialect translation, showing a strong correlation with human judgment. This work contributes to the growing field of Culturally-Aware Machine Translation and highlights the importance of authentic, culturally representative datasets in advancing low-resource translation systems.
%U https://aclanthology.org/2025.abjadnlp-1.13/
%P 114-135

Markdown (Informal)

[Can LLMs Translate Cultural Nuance in Dialects? A Case Study on Lebanese Arabic](https://aclanthology.org/2025.abjadnlp-1.13/) (Yakhni & Chehab, AbjadNLP 2025)

• Can LLMs Translate Cultural Nuance in Dialects? A Case Study on Lebanese Arabic (Yakhni & Chehab, AbjadNLP 2025)

ACL

• Silvana Yakhni and Ali Chehab. 2025. Can LLMs Translate Cultural Nuance in Dialects? A Case Study on Lebanese Arabic. In Proceedings of the 1st Workshop on NLP for Languages Using Arabic Script, pages 114–135, Abu Dhabi, UAE. Association for Computational Linguistics.