<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="emani-r-2025-matrka">
    <titleInfo>
        <title>Mātṛkā: Multilingual Jailbreak Evaluation of Open-Source Large Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Murali</namePart>
        <namePart type="family">Emani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kashyap</namePart>
        <namePart type="given">Manjusha</namePart>
        <namePart type="family">R</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 1st Workshop on Benchmarks, Harmonization, Annotation, and Standardization for Human-Centric AI in Indian Languages (BHASHA 2025)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Arnab</namePart>
            <namePart type="family">Bhattacharya</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Pawan</namePart>
            <namePart type="family">Goyal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Saptarshi</namePart>
            <namePart type="family">Ghosh</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kripabandhu</namePart>
            <namePart type="family">Ghosh</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Mumbai, India</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-313-5</identifier>
    </relatedItem>
    <abstract>Artificial Intelligence (AI) and Large Language Models (LLMs) are increasingly integrated into high-stakes applications, yet their susceptibility to adversarial prompts poses significant security risks. In this work, we introduce Mātṛkā, a framework for systematically evaluating jailbreak vulnerabilities in open-source multilingual LLMs. Using the open-source dataset across nine sensitive categories, we constructed adversarial prompt sets that combine translation, mixed-language encoding, homoglyph signatures, numeric enforcement, and structural variations. Experiments were conducted on state-of-the-art open-source models from Llama, Qwen, GPT-OSS, Mistral, and Gemma families. Our findings highlight transferability of jailbreaks across multiple languages with varying success rates depending on attack design. We provide empirical insights, a novel taxonomy of multilingual jailbreak strategies, and recommendations for enhancing robustness in safety-critical environments.</abstract>
    <identifier type="citekey">emani-r-2025-matrka</identifier>
    <location>
        <url>https://aclanthology.org/2025.bhasha-1.10/</url>
    </location>
    <part>
        <date>2025-12</date>
        <extent unit="page">
            <start>117</start>
            <end>121</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Mātṛkā: Multilingual Jailbreak Evaluation of Open-Source Large Language Models
%A Emani, Murali
%A R, Kashyap Manjusha
%Y Bhattacharya, Arnab
%Y Goyal, Pawan
%Y Ghosh, Saptarshi
%Y Ghosh, Kripabandhu
%S Proceedings of the 1st Workshop on Benchmarks, Harmonization, Annotation, and Standardization for Human-Centric AI in Indian Languages (BHASHA 2025)
%D 2025
%8 December
%I Association for Computational Linguistics
%C Mumbai, India
%@ 979-8-89176-313-5
%F emani-r-2025-matrka
%X Artificial Intelligence (AI) and Large Language Models (LLMs) are increasingly integrated into high-stakes applications, yet their susceptibility to adversarial prompts poses significant security risks. In this work, we introduce Mātṛkā, a framework for systematically evaluating jailbreak vulnerabilities in open-source multilingual LLMs. Using the open-source dataset across nine sensitive categories, we constructed adversarial prompt sets that combine translation, mixed-language encoding, homoglyph signatures, numeric enforcement, and structural variations. Experiments were conducted on state-of-the-art open-source models from Llama, Qwen, GPT-OSS, Mistral, and Gemma families. Our findings highlight transferability of jailbreaks across multiple languages with varying success rates depending on attack design. We provide empirical insights, a novel taxonomy of multilingual jailbreak strategies, and recommendations for enhancing robustness in safety-critical environments.
%U https://aclanthology.org/2025.bhasha-1.10/
%P 117-121

Markdown (Informal)

[Mātṛkā: Multilingual Jailbreak Evaluation of Open-Source Large Language Models](https://aclanthology.org/2025.bhasha-1.10/) (Emani & R, BHASHA 2025)

• Mātṛkā: Multilingual Jailbreak Evaluation of Open-Source Large Language Models (Emani & R, BHASHA 2025)

ACL

• Murali Emani and Kashyap Manjusha R. 2025. Mātṛkā: Multilingual Jailbreak Evaluation of Open-Source Large Language Models. In Proceedings of the 1st Workshop on Benchmarks, Harmonization, Annotation, and Standardization for Human-Centric AI in Indian Languages (BHASHA 2025), pages 117–121, Mumbai, India. Association for Computational Linguistics.