<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kin-lam-etal-2024-compact">
    <titleInfo>
        <title>Compact Speech Translation Models via Discrete Speech Units Pretraining</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Tsz</namePart>
        <namePart type="given">Kin</namePart>
        <namePart type="family">Lam</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Alexandra</namePart>
        <namePart type="family">Birch</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Barry</namePart>
        <namePart type="family">Haddow</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Elizabeth</namePart>
            <namePart type="family">Salesky</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marcello</namePart>
            <namePart type="family">Federico</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marine</namePart>
            <namePart type="family">Carpuat</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand (in-person and online)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We propose a pretraining method to use Self-Supervised Speech (SSS) model to creating more compact Speech-to-text Translation. In contrast to using the SSS model for initialization, our method is more suitable to memory constrained scenario such as on-device deployment. Our method is based on Discrete Speech Units (DSU) extracted from the SSS model. In the first step, our method pretrains two smaller encoder-decoder models on 1) Filterbank-to-DSU (Fbk-to-DSU) and 2) DSU-to-Translation (DSU-to-Trl) data respectively. The DSU thus become the distillation inputs of the smaller models. Subsequently, the encoder from the Fbk-to-DSU model and the decoder from the DSU-to-Trl model are taken to initialise the compact model. Finally, the compact model is finetuned on the paired Fbk-Trl data. In addition to being compact, our method requires no transcripts, making it applicable to low-resource settings. It also avoids speech discretization in inference and is more robust to the DSU tokenization. Evaluation on CoVoST-2 (X-En) shows that our method has consistent improvement over the baseline in three metrics while being compact i.e., only half the SSS model size.</abstract>
    <identifier type="citekey">kin-lam-etal-2024-compact</identifier>
    <identifier type="doi">10.18653/v1/2024.iwslt-1.16</identifier>
    <location>
        <url>https://aclanthology.org/2024.iwslt-1.16</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>114</start>
            <end>124</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Compact Speech Translation Models via Discrete Speech Units Pretraining
%A Lam, Tsz Kin
%A Birch, Alexandra
%A Haddow, Barry
%Y Salesky, Elizabeth
%Y Federico, Marcello
%Y Carpuat, Marine
%S Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024)
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand (in-person and online)
%F kin-lam-etal-2024-compact
%X We propose a pretraining method to use Self-Supervised Speech (SSS) model to creating more compact Speech-to-text Translation. In contrast to using the SSS model for initialization, our method is more suitable to memory constrained scenario such as on-device deployment. Our method is based on Discrete Speech Units (DSU) extracted from the SSS model. In the first step, our method pretrains two smaller encoder-decoder models on 1) Filterbank-to-DSU (Fbk-to-DSU) and 2) DSU-to-Translation (DSU-to-Trl) data respectively. The DSU thus become the distillation inputs of the smaller models. Subsequently, the encoder from the Fbk-to-DSU model and the decoder from the DSU-to-Trl model are taken to initialise the compact model. Finally, the compact model is finetuned on the paired Fbk-Trl data. In addition to being compact, our method requires no transcripts, making it applicable to low-resource settings. It also avoids speech discretization in inference and is more robust to the DSU tokenization. Evaluation on CoVoST-2 (X-En) shows that our method has consistent improvement over the baseline in three metrics while being compact i.e., only half the SSS model size.
%R 10.18653/v1/2024.iwslt-1.16
%U https://aclanthology.org/2024.iwslt-1.16
%U https://doi.org/10.18653/v1/2024.iwslt-1.16
%P 114-124

Markdown (Informal)

[Compact Speech Translation Models via Discrete Speech Units Pretraining](https://aclanthology.org/2024.iwslt-1.16) (Lam et al., IWSLT 2024)

• Compact Speech Translation Models via Discrete Speech Units Pretraining (Lam et al., IWSLT 2024)

ACL

• Tsz Kin Lam, Alexandra Birch, and Barry Haddow. 2024. Compact Speech Translation Models via Discrete Speech Units Pretraining. In Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024), pages 114–124, Bangkok, Thailand (in-person and online). Association for Computational Linguistics.