<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lin-etal-2022-taiwanese">
    <titleInfo>
        <title>Taiwanese-Accented Mandarin and English Multi-Speaker Talking-Face Synthesis System</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Chia-Hsuan</namePart>
        <namePart type="family">Lin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jian-Peng</namePart>
        <namePart type="family">Liao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Cho-Chun</namePart>
        <namePart type="family">Hsieh</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kai-Chun</namePart>
        <namePart type="family">Liao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chun-Hsin</namePart>
        <namePart type="family">Wu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <language>
        <languageTerm type="text">Chinese</languageTerm>
        <languageTerm type="code" authority="iso639-2b">chi</languageTerm>
    </language>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 34th Conference on Computational Linguistics and Speech Processing (ROCLING 2022)</title>
        </titleInfo>
        <originInfo>
            <publisher>The Association for Computational Linguistics and Chinese Language Processing (ACLCLP)</publisher>
            <place>
                <placeTerm type="text">Taipei, Taiwan</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>This paper proposes a multi-speaker talking-face synthesis system. The system incorporates voice cloning and lip-syncing technology to achieve text-to-talking-face generation by acquiring audio and video clips of any speaker and using zero-shot transfer learning. In addition, we used open-source corpora to train several Taiwanese-accented models and proposed using Mandarin Phonetic Symbols (Bopomofo) as the character embedding of the synthesizer to improve the system’s ability to synthesize Chinese-English code-switched sentences. Through our system, users can create rich applications. Also, the research on this technology is novel in the audiovisual speech synthesis field.</abstract>
    <identifier type="citekey">lin-etal-2022-taiwanese</identifier>
    <location>
        <url>https://aclanthology.org/2022.rocling-1.6</url>
    </location>
    <part>
        <date>2022-11</date>
        <extent unit="page">
            <start>40</start>
            <end>48</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Taiwanese-Accented Mandarin and English Multi-Speaker Talking-Face Synthesis System
%A Lin, Chia-Hsuan
%A Liao, Jian-Peng
%A Hsieh, Cho-Chun
%A Liao, Kai-Chun
%A Wu, Chun-Hsin
%S Proceedings of the 34th Conference on Computational Linguistics and Speech Processing (ROCLING 2022)
%D 2022
%8 November
%I The Association for Computational Linguistics and Chinese Language Processing (ACLCLP)
%C Taipei, Taiwan
%G Chinese
%F lin-etal-2022-taiwanese
%X This paper proposes a multi-speaker talking-face synthesis system. The system incorporates voice cloning and lip-syncing technology to achieve text-to-talking-face generation by acquiring audio and video clips of any speaker and using zero-shot transfer learning. In addition, we used open-source corpora to train several Taiwanese-accented models and proposed using Mandarin Phonetic Symbols (Bopomofo) as the character embedding of the synthesizer to improve the system’s ability to synthesize Chinese-English code-switched sentences. Through our system, users can create rich applications. Also, the research on this technology is novel in the audiovisual speech synthesis field.
%U https://aclanthology.org/2022.rocling-1.6
%P 40-48

Markdown (Informal)

[Taiwanese-Accented Mandarin and English Multi-Speaker Talking-Face Synthesis System](https://aclanthology.org/2022.rocling-1.6) (Lin et al., ROCLING 2022)

• Taiwanese-Accented Mandarin and English Multi-Speaker Talking-Face Synthesis System (Lin et al., ROCLING 2022)

ACL

• Chia-Hsuan Lin, Jian-Peng Liao, Cho-Chun Hsieh, Kai-Chun Liao, and Chun-Hsin Wu. 2022. Taiwanese-Accented Mandarin and English Multi-Speaker Talking-Face Synthesis System. In Proceedings of the 34th Conference on Computational Linguistics and Speech Processing (ROCLING 2022), pages 40–48, Taipei, Taiwan. The Association for Computational Linguistics and Chinese Language Processing (ACLCLP).