<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="won-etal-2025-end">
    <titleInfo>
        <title>End-to-End Multilingual Automatic Dubbing via Duration-based Translation with Large Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hyun-Sik</namePart>
        <namePart type="family">Won</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">DongJin</namePart>
        <namePart type="family">Jeong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hyunkyu</namePart>
        <namePart type="family">Choi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jinwon</namePart>
        <namePart type="family">Kim</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Ivan</namePart>
            <namePart type="family">Habernal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Peter</namePart>
            <namePart type="family">Schulam</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jörg</namePart>
            <namePart type="family">Tiedemann</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Suzhou, China</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-334-0</identifier>
    </relatedItem>
    <abstract>Automatic dubbing (AD) aims to replace the original speech in a video with translated speech that maintains precise temporal alignment (isochrony). Achieving natural synchronization between dubbed speech and visual content remains challenging due to variations in speech durations across languages. To address this, we propose an end-to-end AD framework that leverages large language models (LLMs) to integrate translation and timing control seamlessly. At the core of our framework lies Duration-based Translation (DT), a method that dynamically predicts the optimal phoneme count based on source speech duration and iteratively adjusts the translation length accordingly. Our experiments on English, Spanish, and Korean language pairs demonstrate that our approach substantially improves speech overlap—achieving up to 24% relative gains compared to translations without explicit length constraints—while maintaining competitive translation quality measured by COMET scores. Furthermore, our framework does not require language-specific tuning, ensuring practicality for multilingual dubbing scenarios.</abstract>
    <identifier type="citekey">won-etal-2025-end</identifier>
    <location>
        <url>https://aclanthology.org/2025.emnlp-demos.37/</url>
    </location>
    <part>
        <date>2025-11</date>
        <extent unit="page">
            <start>515</start>
            <end>521</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T End-to-End Multilingual Automatic Dubbing via Duration-based Translation with Large Language Models
%A Won, Hyun-Sik
%A Jeong, DongJin
%A Choi, Hyunkyu
%A Kim, Jinwon
%Y Habernal, Ivan
%Y Schulam, Peter
%Y Tiedemann, Jörg
%S Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
%D 2025
%8 November
%I Association for Computational Linguistics
%C Suzhou, China
%@ 979-8-89176-334-0
%F won-etal-2025-end
%X Automatic dubbing (AD) aims to replace the original speech in a video with translated speech that maintains precise temporal alignment (isochrony). Achieving natural synchronization between dubbed speech and visual content remains challenging due to variations in speech durations across languages. To address this, we propose an end-to-end AD framework that leverages large language models (LLMs) to integrate translation and timing control seamlessly. At the core of our framework lies Duration-based Translation (DT), a method that dynamically predicts the optimal phoneme count based on source speech duration and iteratively adjusts the translation length accordingly. Our experiments on English, Spanish, and Korean language pairs demonstrate that our approach substantially improves speech overlap—achieving up to 24% relative gains compared to translations without explicit length constraints—while maintaining competitive translation quality measured by COMET scores. Furthermore, our framework does not require language-specific tuning, ensuring practicality for multilingual dubbing scenarios.
%U https://aclanthology.org/2025.emnlp-demos.37/
%P 515-521

Markdown (Informal)

[End-to-End Multilingual Automatic Dubbing via Duration-based Translation with Large Language Models](https://aclanthology.org/2025.emnlp-demos.37/) (Won et al., EMNLP 2025)

• End-to-End Multilingual Automatic Dubbing via Duration-based Translation with Large Language Models (Won et al., EMNLP 2025)

ACL

• Hyun-Sik Won, DongJin Jeong, Hyunkyu Choi, and Jinwon Kim. 2025. End-to-End Multilingual Automatic Dubbing via Duration-based Translation with Large Language Models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 515–521, Suzhou, China. Association for Computational Linguistics.