<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="sterz-etal-2026-text4speech">
    <titleInfo>
        <title>Text4Speech: Text-Driven Training for Massively Multilingual Speech Understanding</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hannah</namePart>
        <namePart type="family">Sterz</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Anna</namePart>
        <namePart type="family">Korhonen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ivan</namePart>
        <namePart type="family">Vulić</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Goran</namePart>
        <namePart type="family">Glavaš</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Speech is a primary communication interface for humans, and thus a crucial modality for large language models (LLMs). Despite this, the vast majority of open speech-capable LLMs are developed for and evaluated on a small set of \ensuremath&lt;20 high-resource languages, adding to the already massive inequity of modern language technology. Massively multilingual speech understanding is primarily hindered by the limited amounts of speech training data for most languages and tasks. For the lion’s share of the world’s languages, speech datasets for tasks other than automatic speech recognition (ASR) simply do not exist. In this work, we bypass the sparsity of task-specific speech data by leveraging text – much more available for many languages – to enable an LLM to understand speech input. Our approach, dubbed Text4Speech, leverages state-of-the-art multilingual pre-aligned text and speech encoders: we first align the multilingual text encoder to an LLM using abundant text data and then further align the speech encoder to the LLM using a limited amount of speech. In both steps, we train the alignment parameters via self-distillation on the LLM. Our evaluation on three multilingual speech understanding benchmarks that encompass 90 languages renders Text4Speech highly effective, yielding average improvements of up to 6% on Fleurs-SIB over a very strong ASR-based cascade baseline. Crucially, we show that our text-based alignment unlocks a highly sample-efficient alignment of the speech encoder to the LLM.</abstract>
    <identifier type="citekey">sterz-etal-2026-text4speech</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.2427/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>55111</start>
            <end>55126</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Text4Speech: Text-Driven Training for Massively Multilingual Speech Understanding
%A Sterz, Hannah
%A Korhonen, Anna
%A Vulić, Ivan
%A Glavaš, Goran
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F sterz-etal-2026-text4speech
%X Speech is a primary communication interface for humans, and thus a crucial modality for large language models (LLMs). Despite this, the vast majority of open speech-capable LLMs are developed for and evaluated on a small set of \ensuremath<20 high-resource languages, adding to the already massive inequity of modern language technology. Massively multilingual speech understanding is primarily hindered by the limited amounts of speech training data for most languages and tasks. For the lion’s share of the world’s languages, speech datasets for tasks other than automatic speech recognition (ASR) simply do not exist. In this work, we bypass the sparsity of task-specific speech data by leveraging text – much more available for many languages – to enable an LLM to understand speech input. Our approach, dubbed Text4Speech, leverages state-of-the-art multilingual pre-aligned text and speech encoders: we first align the multilingual text encoder to an LLM using abundant text data and then further align the speech encoder to the LLM using a limited amount of speech. In both steps, we train the alignment parameters via self-distillation on the LLM. Our evaluation on three multilingual speech understanding benchmarks that encompass 90 languages renders Text4Speech highly effective, yielding average improvements of up to 6% on Fleurs-SIB over a very strong ASR-based cascade baseline. Crucially, we show that our text-based alignment unlocks a highly sample-efficient alignment of the speech encoder to the LLM.
%U https://aclanthology.org/2026.emnlp-main.2427/
%P 55111-55126

Markdown (Informal)

[Text4Speech: Text-Driven Training for Massively Multilingual Speech Understanding](https://aclanthology.org/2026.emnlp-main.2427/) (Sterz et al., EMNLP 2026)

• Text4Speech: Text-Driven Training for Massively Multilingual Speech Understanding (Sterz et al., EMNLP 2026)

ACL

• Hannah Sterz, Anna Korhonen, Ivan Vulić, and Goran Glavaš. 2026. Text4Speech: Text-Driven Training for Massively Multilingual Speech Understanding. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 55111–55126, Budapest, Hungary. Association for Computational Linguistics.