<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="fukuda-etal-2021-knowledge">
    <titleInfo>
        <title>On Knowledge Distillation for Translating Erroneous Speech Transcriptions</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Ryo</namePart>
        <namePart type="family">Fukuda</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Katsuhito</namePart>
        <namePart type="family">Sudoh</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Satoshi</namePart>
        <namePart type="family">Nakamura</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Marcello</namePart>
            <namePart type="family">Federico</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Alex</namePart>
            <namePart type="family">Waibel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marta</namePart>
            <namePart type="given">R</namePart>
            <namePart type="family">Costa-jussà</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jan</namePart>
            <namePart type="family">Niehues</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sebastian</namePart>
            <namePart type="family">Stuker</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Elizabeth</namePart>
            <namePart type="family">Salesky</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand (online)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Recent studies argue that knowledge distillation is promising for speech translation (ST) using end-to-end models. In this work, we investigate the effect of knowledge distillation with a cascade ST using automatic speech recognition (ASR) and machine translation (MT) models. We distill knowledge from a teacher model based on human transcripts to a student model based on erroneous transcriptions. Our experimental results demonstrated that knowledge distillation is beneficial for a cascade ST. Further investigation that combined knowledge distillation and fine-tuning revealed that the combination consistently improved two language pairs: English-Italian and Spanish-English.</abstract>
    <identifier type="citekey">fukuda-etal-2021-knowledge</identifier>
    <identifier type="doi">10.18653/v1/2021.iwslt-1.24</identifier>
    <location>
        <url>https://aclanthology.org/2021.iwslt-1.24</url>
    </location>
    <part>
        <date>2021-08</date>
        <extent unit="page">
            <start>198</start>
            <end>205</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T On Knowledge Distillation for Translating Erroneous Speech Transcriptions
%A Fukuda, Ryo
%A Sudoh, Katsuhito
%A Nakamura, Satoshi
%Y Federico, Marcello
%Y Waibel, Alex
%Y Costa-jussà, Marta R.
%Y Niehues, Jan
%Y Stuker, Sebastian
%Y Salesky, Elizabeth
%S Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021)
%D 2021
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand (online)
%F fukuda-etal-2021-knowledge
%X Recent studies argue that knowledge distillation is promising for speech translation (ST) using end-to-end models. In this work, we investigate the effect of knowledge distillation with a cascade ST using automatic speech recognition (ASR) and machine translation (MT) models. We distill knowledge from a teacher model based on human transcripts to a student model based on erroneous transcriptions. Our experimental results demonstrated that knowledge distillation is beneficial for a cascade ST. Further investigation that combined knowledge distillation and fine-tuning revealed that the combination consistently improved two language pairs: English-Italian and Spanish-English.
%R 10.18653/v1/2021.iwslt-1.24
%U https://aclanthology.org/2021.iwslt-1.24
%U https://doi.org/10.18653/v1/2021.iwslt-1.24
%P 198-205

Markdown (Informal)

[On Knowledge Distillation for Translating Erroneous Speech Transcriptions](https://aclanthology.org/2021.iwslt-1.24) (Fukuda et al., IWSLT 2021)

• On Knowledge Distillation for Translating Erroneous Speech Transcriptions (Fukuda et al., IWSLT 2021)

ACL

• Ryo Fukuda, Katsuhito Sudoh, and Satoshi Nakamura. 2021. On Knowledge Distillation for Translating Erroneous Speech Transcriptions. In Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021), pages 198–205, Bangkok, Thailand (online). Association for Computational Linguistics.