<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="arenas-manrique-2026-probing">
    <titleInfo>
        <title>Probing Language Understanding in Colombian Indigenous Languages: Zero-Shot Translation, Visual Grounding, and Cultural Fidelity</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Santiago</namePart>
        <namePart type="family">Arenas</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ruben</namePart>
        <namePart type="family">Manrique</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 3rd Workshop on Language Understanding in the Human-Machine Era (LUHME 2026)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Rui</namePart>
            <namePart type="family">Sousa-Silva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Henrique</namePart>
            <namePart type="family">Lopes Cardoso</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Maarit</namePart>
            <namePart type="family">Koponen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Antonio</namePart>
            <namePart type="family">Pareja-Lora</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-471-2</identifier>
    </relatedItem>
    <abstract>Language models can generate fluent, confident-sounding text even for languages they barely cover, but fluency alone reveals little about whether that text preserves situated meaning, cultural reference, and communicative intent. We study this question for Colombian Indigenous languages through two complementary probes of language understanding: (1)zero-shot machine translation across five languages (Arhuaco, Inga, Kamentsá, Nasa Yuwe, Wayuunaiki) and (2)multimodal cultural grounding on a community-built Inga image-description dataset that, to our knowledge, is the largest available for Inga. We additionally document and release this culturally grounded multimodal dataset as a reusable resource for the Inga community and future research. We evaluate GPT-5.2, GPT-4o-mini, Gemini 2.5 Flash-Lite, Qwen2-VL-72B-Instruct, NLLB-200, and T5 variants without fine-tuning. GPT-5.2 achieves the highest translation scores (chrF 21.36, BLEU 3.42), yet absolute performance remains uniformly low. No evaluated model is suitable for direct deployment, placing Colombian Indigenous languages among the most underserved in NLP even relative to other low-resource languages. For multimodal cultural grounding, semantically aligned visual context consistently improves preservation of community-provided cultural ideas, while mismatched images cause sharp degradation, a distinction that standard similarity metrics fail to capture. These findings show that fluent generation can mask failures of situated language understanding and motivate purpose-built evaluation for Indigenous language NLP.</abstract>
    <identifier type="citekey">arenas-manrique-2026-probing</identifier>
    <location>
        <url>https://aclanthology.org/2026.luhme-1.11/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>121</start>
            <end>133</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Probing Language Understanding in Colombian Indigenous Languages: Zero-Shot Translation, Visual Grounding, and Cultural Fidelity
%A Arenas, Santiago
%A Manrique, Ruben
%Y Sousa-Silva, Rui
%Y Lopes Cardoso, Henrique
%Y Koponen, Maarit
%Y Pareja-Lora, Antonio
%S Proceedings of the 3rd Workshop on Language Understanding in the Human-Machine Era (LUHME 2026)
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-471-2
%F arenas-manrique-2026-probing
%X Language models can generate fluent, confident-sounding text even for languages they barely cover, but fluency alone reveals little about whether that text preserves situated meaning, cultural reference, and communicative intent. We study this question for Colombian Indigenous languages through two complementary probes of language understanding: (1)zero-shot machine translation across five languages (Arhuaco, Inga, Kamentsá, Nasa Yuwe, Wayuunaiki) and (2)multimodal cultural grounding on a community-built Inga image-description dataset that, to our knowledge, is the largest available for Inga. We additionally document and release this culturally grounded multimodal dataset as a reusable resource for the Inga community and future research. We evaluate GPT-5.2, GPT-4o-mini, Gemini 2.5 Flash-Lite, Qwen2-VL-72B-Instruct, NLLB-200, and T5 variants without fine-tuning. GPT-5.2 achieves the highest translation scores (chrF 21.36, BLEU 3.42), yet absolute performance remains uniformly low. No evaluated model is suitable for direct deployment, placing Colombian Indigenous languages among the most underserved in NLP even relative to other low-resource languages. For multimodal cultural grounding, semantically aligned visual context consistently improves preservation of community-provided cultural ideas, while mismatched images cause sharp degradation, a distinction that standard similarity metrics fail to capture. These findings show that fluent generation can mask failures of situated language understanding and motivate purpose-built evaluation for Indigenous language NLP.
%U https://aclanthology.org/2026.luhme-1.11/
%P 121-133

Markdown (Informal)

[Probing Language Understanding in Colombian Indigenous Languages: Zero-Shot Translation, Visual Grounding, and Cultural Fidelity](https://aclanthology.org/2026.luhme-1.11/) (Arenas & Manrique, LUHME 2026)

• Probing Language Understanding in Colombian Indigenous Languages: Zero-Shot Translation, Visual Grounding, and Cultural Fidelity (Arenas & Manrique, LUHME 2026)

ACL

• Santiago Arenas and Ruben Manrique. 2026. Probing Language Understanding in Colombian Indigenous Languages: Zero-Shot Translation, Visual Grounding, and Cultural Fidelity. In Proceedings of the 3rd Workshop on Language Understanding in the Human-Machine Era (LUHME 2026), pages 121–133, Budapest, Hungary. Association for Computational Linguistics.