<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="zhao-etal-2023-generating">
    <titleInfo>
        <title>Generating Synthetic Speech from SpokenVocab for Speech Translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Jinming</namePart>
        <namePart type="family">Zhao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Gholamreza</namePart>
        <namePart type="family">Haffari</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ehsan</namePart>
        <namePart type="family">Shareghi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Andreas</namePart>
            <namePart type="family">Vlachos</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Isabelle</namePart>
            <namePart type="family">Augenstein</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Dubrovnik, Croatia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Training end-to-end speech translation (ST) systems requires sufficiently large-scale data, which is unavailable for most language pairs and domains. One practical solution to the data scarcity issue is to convert text-based machine translation (MT) data to ST data via text-to-speech (TTS) systems. Yet, using TTS systems can be tedious and slow. In this work, we propose SpokenVocab, a simple, scalable and effective data augmentation technique to convert MT data to ST data on-the-fly. The idea is to retrieve and stitch audio snippets, corresponding to words in an MT sentence, from a spoken vocabulary bank. Our experiments on multiple language pairs show that stitched speech helps to improve translation quality by an average of 1.83 BLEU score, while performing equally well as TTS-generated speech in improving translation quality. We also showcase how SpokenVocab can be applied in code-switching ST for which often no TTS systems exit.</abstract>
    <identifier type="citekey">zhao-etal-2023-generating</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-eacl.147</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-eacl.147</url>
    </location>
    <part>
        <date>2023-05</date>
        <extent unit="page">
            <start>1975</start>
            <end>1981</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Generating Synthetic Speech from SpokenVocab for Speech Translation
%A Zhao, Jinming
%A Haffari, Gholamreza
%A Shareghi, Ehsan
%Y Vlachos, Andreas
%Y Augenstein, Isabelle
%S Findings of the Association for Computational Linguistics: EACL 2023
%D 2023
%8 May
%I Association for Computational Linguistics
%C Dubrovnik, Croatia
%F zhao-etal-2023-generating
%X Training end-to-end speech translation (ST) systems requires sufficiently large-scale data, which is unavailable for most language pairs and domains. One practical solution to the data scarcity issue is to convert text-based machine translation (MT) data to ST data via text-to-speech (TTS) systems. Yet, using TTS systems can be tedious and slow. In this work, we propose SpokenVocab, a simple, scalable and effective data augmentation technique to convert MT data to ST data on-the-fly. The idea is to retrieve and stitch audio snippets, corresponding to words in an MT sentence, from a spoken vocabulary bank. Our experiments on multiple language pairs show that stitched speech helps to improve translation quality by an average of 1.83 BLEU score, while performing equally well as TTS-generated speech in improving translation quality. We also showcase how SpokenVocab can be applied in code-switching ST for which often no TTS systems exit.
%R 10.18653/v1/2023.findings-eacl.147
%U https://aclanthology.org/2023.findings-eacl.147
%U https://doi.org/10.18653/v1/2023.findings-eacl.147
%P 1975-1981

Markdown (Informal)

[Generating Synthetic Speech from SpokenVocab for Speech Translation](https://aclanthology.org/2023.findings-eacl.147) (Zhao et al., Findings 2023)

• Generating Synthetic Speech from SpokenVocab for Speech Translation (Zhao et al., Findings 2023)

ACL

• Jinming Zhao, Gholamreza Haffari, and Ehsan Shareghi. 2023. Generating Synthetic Speech from SpokenVocab for Speech Translation. In Findings of the Association for Computational Linguistics: EACL 2023, pages 1975–1981, Dubrovnik, Croatia. Association for Computational Linguistics.