<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="nazeem-etal-2026-foundational">
    <titleInfo>
        <title>A Foundational Study of Multimodal Speech–Text Alignment in Malayalam</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Meharuniza</namePart>
        <namePart type="family">Nazeem</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Navaneeth</namePart>
        <namePart type="family">S</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Adithyakiran</namePart>
        <namePart type="family">A S</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Rajeev</namePart>
        <namePart type="family">R R</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ashbi</namePart>
        <namePart type="family">Biju</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Fifth Workshop on Multimodal Machine Learning in Low-resource Languages</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Rahul</namePart>
            <namePart type="family">Ponnusamy</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Prasanna</namePart>
            <namePart type="given">Kumar</namePart>
            <namePart type="family">Kumaresan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Saranya</namePart>
            <namePart type="family">Rajiakodi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Premjith</namePart>
            <namePart type="family">B</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Shunmuga</namePart>
            <namePart type="given">Priya</namePart>
            <namePart type="family">MC</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Abirami</namePart>
            <namePart type="family">Murugappan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jyothish</namePart>
            <namePart type="given">Lal</namePart>
            <namePart type="family">G</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sowmya</namePart>
            <namePart type="family">V</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Bharathi</namePart>
            <namePart type="given">Raja</namePart>
            <namePart type="family">Chakravarthi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Hengqin, China</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-445-3</identifier>
    </relatedItem>
    <abstract>Speech–text retrieval is a multimodal alignment task in which spoken audio and written transcripts are mapped into a shared representation space. Standard retrieval metrics test whether an audio embedding recovers its transcript from a large candidate pool, but not whether the shared representation preserves small grammatical contrasts inside agglutinated words. We construct an automatic Malayalam benchmark with 12,611 controlled audio–text comparison rows derived from 6,611 held-out audio queries. The benchmark contains three morphosyntactic categories—case, tense, and negation—and a lexical numeral-control category. Alternatives are generated with the mlmorph finitestate analyser and generator and retained after a generate–reanalyse consistency check. A CLIP-style model without explicit hard negatives obtains 37.45% full-pool Recall@1 but only 53.66% pairwise accuracy, close to the 50% pairwise baseline. We diagnose a fixedfirst-candidate loader defect that under-delivers later-generated categories and replace it with per-transcript category-aware, access-time resampling. At the matched epoch count, pairwise accuracy rises from 73.50% to 79.76%, and tense accuracy rises from 52.83% to 77.70%. The complete corrected procedure reaches 89.69% pairwise accuracy after extended training, while full-pool retrieval remains below the original checkpoint. A stratified human review of 797 rows retains or corrects 731 (91.7%); the final checkpoint obtains 74.01% on this deliberately reweighted and partly rewritten subset. These results show that global retrieval, controlled grammatical discrimination, candidate-delivery logic, and human benchmark validation are distinct components of multimodal evaluation.</abstract>
    <identifier type="citekey">nazeem-etal-2026-foundational</identifier>
    <location>
        <url>https://aclanthology.org/2026.mmlow-main.2/</url>
    </location>
    <part>
        <date>2026-11</date>
        <extent unit="page">
            <start>18</start>
            <end>27</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T A Foundational Study of Multimodal Speech–Text Alignment in Malayalam
%A Nazeem, Meharuniza
%A S, Navaneeth
%A A S, Adithyakiran
%A R R, Rajeev
%A Biju, Ashbi
%Y Ponnusamy, Rahul
%Y Kumaresan, Prasanna Kumar
%Y Rajiakodi, Saranya
%Y B, Premjith
%Y MC, Shunmuga Priya
%Y Murugappan, Abirami
%Y G, Jyothish Lal
%Y V, Sowmya
%Y Chakravarthi, Bharathi Raja
%S Proceedings of the Fifth Workshop on Multimodal Machine Learning in Low-resource Languages
%D 2026
%8 November
%I Association for Computational Linguistics
%C Hengqin, China
%@ 979-8-89176-445-3
%F nazeem-etal-2026-foundational
%X Speech–text retrieval is a multimodal alignment task in which spoken audio and written transcripts are mapped into a shared representation space. Standard retrieval metrics test whether an audio embedding recovers its transcript from a large candidate pool, but not whether the shared representation preserves small grammatical contrasts inside agglutinated words. We construct an automatic Malayalam benchmark with 12,611 controlled audio–text comparison rows derived from 6,611 held-out audio queries. The benchmark contains three morphosyntactic categories—case, tense, and negation—and a lexical numeral-control category. Alternatives are generated with the mlmorph finitestate analyser and generator and retained after a generate–reanalyse consistency check. A CLIP-style model without explicit hard negatives obtains 37.45% full-pool Recall@1 but only 53.66% pairwise accuracy, close to the 50% pairwise baseline. We diagnose a fixedfirst-candidate loader defect that under-delivers later-generated categories and replace it with per-transcript category-aware, access-time resampling. At the matched epoch count, pairwise accuracy rises from 73.50% to 79.76%, and tense accuracy rises from 52.83% to 77.70%. The complete corrected procedure reaches 89.69% pairwise accuracy after extended training, while full-pool retrieval remains below the original checkpoint. A stratified human review of 797 rows retains or corrects 731 (91.7%); the final checkpoint obtains 74.01% on this deliberately reweighted and partly rewritten subset. These results show that global retrieval, controlled grammatical discrimination, candidate-delivery logic, and human benchmark validation are distinct components of multimodal evaluation.
%U https://aclanthology.org/2026.mmlow-main.2/
%P 18-27

Markdown (Informal)

[A Foundational Study of Multimodal Speech–Text Alignment in Malayalam](https://aclanthology.org/2026.mmlow-main.2/) (Nazeem et al., MMLow 2026)

• A Foundational Study of Multimodal Speech–Text Alignment in Malayalam (Nazeem et al., MMLow 2026)

ACL

• Meharuniza Nazeem, Navaneeth S, Adithyakiran A S, Rajeev R R, and Ashbi Biju. 2026. A Foundational Study of Multimodal Speech–Text Alignment in Malayalam. In Proceedings of the Fifth Workshop on Multimodal Machine Learning in Low-resource Languages, pages 18–27, Hengqin, China. Association for Computational Linguistics.