<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="modica-etal-2026-factual">
    <titleInfo>
        <title>Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models?</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Luca</namePart>
        <namePart type="family">Modica</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Filip</namePart>
        <namePart type="family">Landin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mehrdad</namePart>
        <namePart type="family">Farahani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Livia</namePart>
        <namePart type="family">Qian</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Gabriel</namePart>
        <namePart type="family">Skantze</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Richard</namePart>
        <namePart type="family">Johansson</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Saif</namePart>
            <namePart type="given">M</namePart>
            <namePart type="family">Mohammad</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nedjma</namePart>
            <namePart type="family">Ousidhoum</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">San Diego, California, United States</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-413-2</identifier>
    </relatedItem>
    <abstract>In recent years, several Speech Language Models (SLMs) that represent speech and written text jointly have been presented. The question then emerges about how model-internal mechanisms are similar and different when operating in the two modalities. We focus on how these systems encode, store, and retrieve factual knowledge, which has previously been investigated for text-only models. To investigate mechanisms behind the storage and recall of factual association in SLMs, we leverage Causal Mediation Analysis, a technique previously applied to text-based models. Initial results using SpiritLM, a multimodal model integrating discrete speech tokens reveal discrepancies between text-to-text and speech-to-text results, suggesting that the emergent mechanisms for factual recall are only partially carried over from the text to the speech modality. These results advance our understanding of how internal mechanisms encode factual associations in SLMs while contributing insights for improving speech-enabled AI systems.</abstract>
    <identifier type="citekey">modica-etal-2026-factual</identifier>
    <location>
        <url>https://aclanthology.org/2026.starsem-conference.28/</url>
    </location>
    <part>
        <date>2026-07</date>
        <extent unit="page">
            <start>401</start>
            <end>409</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models?
%A Modica, Luca
%A Landin, Filip
%A Farahani, Mehrdad
%A Qian, Livia
%A Skantze, Gabriel
%A Johansson, Richard
%Y Mohammad, Saif M.
%Y Ousidhoum, Nedjma
%S Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)
%D 2026
%8 July
%I Association for Computational Linguistics
%C San Diego, California, United States
%@ 979-8-89176-413-2
%F modica-etal-2026-factual
%X In recent years, several Speech Language Models (SLMs) that represent speech and written text jointly have been presented. The question then emerges about how model-internal mechanisms are similar and different when operating in the two modalities. We focus on how these systems encode, store, and retrieve factual knowledge, which has previously been investigated for text-only models. To investigate mechanisms behind the storage and recall of factual association in SLMs, we leverage Causal Mediation Analysis, a technique previously applied to text-based models. Initial results using SpiritLM, a multimodal model integrating discrete speech tokens reveal discrepancies between text-to-text and speech-to-text results, suggesting that the emergent mechanisms for factual recall are only partially carried over from the text to the speech modality. These results advance our understanding of how internal mechanisms encode factual associations in SLMs while contributing insights for improving speech-enabled AI systems.
%U https://aclanthology.org/2026.starsem-conference.28/
%P 401-409

Markdown (Informal)

[Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models?](https://aclanthology.org/2026.starsem-conference.28/) (Modica et al., *SEM 2026)

• Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models? (Modica et al., *SEM 2026)

ACL

• Luca Modica, Filip Landin, Mehrdad Farahani, Livia Qian, Gabriel Skantze, and Richard Johansson. 2026. Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models?. In Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026), pages 401–409, San Diego, California, United States. Association for Computational Linguistics.