<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="bhattacharjee-etal-2024-domain-dynamics">
    <titleInfo>
        <title>Domain Dynamics: Evaluating Large Language Models in English-Hindi Translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Soham</namePart>
        <namePart type="family">Bhattacharjee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Baban</namePart>
        <namePart type="family">Gain</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Asif</namePart>
        <namePart type="family">Ekbal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 21st International Conference on Natural Language Processing (ICON)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Sobha</namePart>
            <namePart type="family">Lalitha Devi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Karunesh</namePart>
            <namePart type="family">Arora</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>NLP Association of India (NLPAI)</publisher>
            <place>
                <placeTerm type="text">AU-KBC Research Centre, Chennai, India</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Large Language Models (LLMs) have demonstrated impressive capabilities in machine translation, leveraging extensive pre-training on vast amounts of data. However, this generalist training often overlooks domain-specific nuances, leading to potential difficulties when translating specialized texts. In this study, we present a multi-domain test suite, collated from previously published datasets, designed to challenge and evaluate the translation abilities of LLMs. The test suite encompasses diverse domains such as judicial, education, literature (specifically religious texts), and noisy user-generated content from online product reviews and forums like Reddit. Each domain consists of approximately 250-300 sentences, carefully curated and randomized in the final compilation. This English-to-Hindi dataset aims to evaluate and expose the limitations of LLM-based translation systems, offering valuable insights into areas requiring further research and development. We have submitted the dataset to WMT24 Break the LLM</abstract>
    <identifier type="citekey">bhattacharjee-etal-2024-domain-dynamics</identifier>
    <location>
        <url>https://aclanthology.org/2024.icon-1.19/</url>
    </location>
    <part>
        <date>2024-12</date>
        <extent unit="page">
            <start>169</start>
            <end>177</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Domain Dynamics: Evaluating Large Language Models in English-Hindi Translation
%A Bhattacharjee, Soham
%A Gain, Baban
%A Ekbal, Asif
%Y Lalitha Devi, Sobha
%Y Arora, Karunesh
%S Proceedings of the 21st International Conference on Natural Language Processing (ICON)
%D 2024
%8 December
%I NLP Association of India (NLPAI)
%C AU-KBC Research Centre, Chennai, India
%F bhattacharjee-etal-2024-domain-dynamics
%X Large Language Models (LLMs) have demonstrated impressive capabilities in machine translation, leveraging extensive pre-training on vast amounts of data. However, this generalist training often overlooks domain-specific nuances, leading to potential difficulties when translating specialized texts. In this study, we present a multi-domain test suite, collated from previously published datasets, designed to challenge and evaluate the translation abilities of LLMs. The test suite encompasses diverse domains such as judicial, education, literature (specifically religious texts), and noisy user-generated content from online product reviews and forums like Reddit. Each domain consists of approximately 250-300 sentences, carefully curated and randomized in the final compilation. This English-to-Hindi dataset aims to evaluate and expose the limitations of LLM-based translation systems, offering valuable insights into areas requiring further research and development. We have submitted the dataset to WMT24 Break the LLM
%U https://aclanthology.org/2024.icon-1.19/
%P 169-177

Markdown (Informal)

[Domain Dynamics: Evaluating Large Language Models in English-Hindi Translation](https://aclanthology.org/2024.icon-1.19/) (Bhattacharjee et al., ICON 2024)

• Domain Dynamics: Evaluating Large Language Models in English-Hindi Translation (Bhattacharjee et al., ICON 2024)

ACL

• Soham Bhattacharjee, Baban Gain, and Asif Ekbal. 2024. Domain Dynamics: Evaluating Large Language Models in English-Hindi Translation. In Proceedings of the 21st International Conference on Natural Language Processing (ICON), pages 169–177, AU-KBC Research Centre, Chennai, India. NLP Association of India (NLPAI).