<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kondo-tamura-2024-inter">
    <titleInfo>
        <title>Inter-language Transfer Learning for Visual Speech Recognition toward Under-resourced Environments</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Fumiya</namePart>
        <namePart type="family">Kondo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Satoshi</namePart>
        <namePart type="family">Tamura</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Maite</namePart>
            <namePart type="family">Melero</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sakriani</namePart>
            <namePart type="family">Sakti</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Claudia</namePart>
            <namePart type="family">Soria</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA and ICCL</publisher>
            <place>
                <placeTerm type="text">Torino, Italia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this study, we introduce a method of inter-language transfer learning for under-resourced visual speech recognition. Deploying speech-related technology to all languages is a quite important activity. However, applying state-of-the-art deep-learning techniques requires huge-size labeled corpora, which makes it hard for under-resourced languages. Our approach leverages a small amount of labeled video data of the target language, and employs inter-language transfer learning using a pre-trained English lip-reading model. By applying the proposed scheme, we build a Japanese lip-reading model, using the ROHAN corpus, the size of which is about one 450th of the size of English datasets. The front-end encoder part of the pre-trained model is fine-tuned to improve the acquisition of pronunciation and lip movement patterns unique to Japanese. On the other hand, the back-end encoder and the decoder are built using the Japanese dataset. Although English and Japanese have different language structures, evaluation experiments show that it is possible to build the Japanese lip-reading model efficiently. Comparison with competitive schemes demonstrates the effectiveness of our method.</abstract>
    <identifier type="citekey">kondo-tamura-2024-inter</identifier>
    <location>
        <url>https://aclanthology.org/2024.sigul-1.19</url>
    </location>
    <part>
        <date>2024-05</date>
        <extent unit="page">
            <start>149</start>
            <end>154</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Inter-language Transfer Learning for Visual Speech Recognition toward Under-resourced Environments
%A Kondo, Fumiya
%A Tamura, Satoshi
%Y Melero, Maite
%Y Sakti, Sakriani
%Y Soria, Claudia
%S Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024
%D 2024
%8 May
%I ELRA and ICCL
%C Torino, Italia
%F kondo-tamura-2024-inter
%X In this study, we introduce a method of inter-language transfer learning for under-resourced visual speech recognition. Deploying speech-related technology to all languages is a quite important activity. However, applying state-of-the-art deep-learning techniques requires huge-size labeled corpora, which makes it hard for under-resourced languages. Our approach leverages a small amount of labeled video data of the target language, and employs inter-language transfer learning using a pre-trained English lip-reading model. By applying the proposed scheme, we build a Japanese lip-reading model, using the ROHAN corpus, the size of which is about one 450th of the size of English datasets. The front-end encoder part of the pre-trained model is fine-tuned to improve the acquisition of pronunciation and lip movement patterns unique to Japanese. On the other hand, the back-end encoder and the decoder are built using the Japanese dataset. Although English and Japanese have different language structures, evaluation experiments show that it is possible to build the Japanese lip-reading model efficiently. Comparison with competitive schemes demonstrates the effectiveness of our method.
%U https://aclanthology.org/2024.sigul-1.19
%P 149-154

Markdown (Informal)

[Inter-language Transfer Learning for Visual Speech Recognition toward Under-resourced Environments](https://aclanthology.org/2024.sigul-1.19) (Kondo & Tamura, SIGUL-WS 2024)

• Inter-language Transfer Learning for Visual Speech Recognition toward Under-resourced Environments (Kondo & Tamura, SIGUL-WS 2024)

ACL

• Fumiya Kondo and Satoshi Tamura. 2024. Inter-language Transfer Learning for Visual Speech Recognition toward Under-resourced Environments. In Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024, pages 149–154, Torino, Italia. ELRA and ICCL.