<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="trinley-etal-2025-language">
    <titleInfo>
        <title>What Language(s) Does Aya-23 Think In? How Multilinguality Affects Internal Language Representations</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Katharina</namePart>
        <namePart type="given">A</namePart>
        <namePart type="given">T</namePart>
        <namePart type="given">T</namePart>
        <namePart type="family">Trinley</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Toshiki</namePart>
        <namePart type="family">Nakai</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Tatiana</namePart>
        <namePart type="family">Anikina</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Tanja</namePart>
        <namePart type="family">Baeumel</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Workshop on Beyond English: Natural Language Processing for all Languages in an Era of Large Language Models</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Sudhansu</namePart>
            <namePart type="given">Bala</namePart>
            <namePart type="family">Das</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Pruthwik</namePart>
            <namePart type="family">Mishra</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Alok</namePart>
            <namePart type="family">Singh</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Shamsuddeen</namePart>
            <namePart type="given">Hassan</namePart>
            <namePart type="family">Muhammad</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Asif</namePart>
            <namePart type="family">Ekbal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Uday</namePart>
            <namePart type="given">Kumar</namePart>
            <namePart type="family">Das</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>INCOMA Ltd., Shoumen, BULGARIA</publisher>
            <place>
                <placeTerm type="text">Varna, Bulgaria</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Large language models (LLMs) excel at multilingual tasks, yet their internal language processing remains poorly understood. We analyze how Aya-23-8B, a decoder-only LLM trained on balanced multilingual data, handles code-mixed, cloze, and translation tasks compared to predominantly monolingual models like Llama 3 and Chinese-LLaMA-2. Using logit lens and neuron specialization analyses, we find: (1) Aya-23 activates typologically related language representations during translation, unlike English-centric models that rely on a single pivot language; (2) code-mixed neuron activation patterns vary with mixing rates and are shaped more by the base language than the mixed-in one; and (3) Aya-23’s language-specific neurons for code-mixed inputs concentrate in final layers, diverging from prior findings on decoder-only models. Neuron overlap analysis further shows that script similarity and typological relations impact processing across model types. These findings reveal how multilingual training shapes LLM internals and inform future cross-lingual transfer research. The code and dataset are publicly available.</abstract>
    <identifier type="citekey">trinley-etal-2025-language</identifier>
    <location>
        <url>https://aclanthology.org/2025.globalnlp-1.18/</url>
    </location>
    <part>
        <date>2025-09</date>
        <extent unit="page">
            <start>159</start>
            <end>171</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T What Language(s) Does Aya-23 Think In? How Multilinguality Affects Internal Language Representations
%A Trinley, Katharina A. T. T.
%A Nakai, Toshiki
%A Anikina, Tatiana
%A Baeumel, Tanja
%Y Das, Sudhansu Bala
%Y Mishra, Pruthwik
%Y Singh, Alok
%Y Muhammad, Shamsuddeen Hassan
%Y Ekbal, Asif
%Y Das, Uday Kumar
%S Proceedings of the Workshop on Beyond English: Natural Language Processing for all Languages in an Era of Large Language Models
%D 2025
%8 September
%I INCOMA Ltd., Shoumen, BULGARIA
%C Varna, Bulgaria
%F trinley-etal-2025-language
%X Large language models (LLMs) excel at multilingual tasks, yet their internal language processing remains poorly understood. We analyze how Aya-23-8B, a decoder-only LLM trained on balanced multilingual data, handles code-mixed, cloze, and translation tasks compared to predominantly monolingual models like Llama 3 and Chinese-LLaMA-2. Using logit lens and neuron specialization analyses, we find: (1) Aya-23 activates typologically related language representations during translation, unlike English-centric models that rely on a single pivot language; (2) code-mixed neuron activation patterns vary with mixing rates and are shaped more by the base language than the mixed-in one; and (3) Aya-23’s language-specific neurons for code-mixed inputs concentrate in final layers, diverging from prior findings on decoder-only models. Neuron overlap analysis further shows that script similarity and typological relations impact processing across model types. These findings reveal how multilingual training shapes LLM internals and inform future cross-lingual transfer research. The code and dataset are publicly available.
%U https://aclanthology.org/2025.globalnlp-1.18/
%P 159-171

Markdown (Informal)

[What Language(s) Does Aya-23 Think In? How Multilinguality Affects Internal Language Representations](https://aclanthology.org/2025.globalnlp-1.18/) (Trinley et al., GlobalNLP 2025)

• What Language(s) Does Aya-23 Think In? How Multilinguality Affects Internal Language Representations (Trinley et al., GlobalNLP 2025)

ACL

• Katharina A. T. T. Trinley, Toshiki Nakai, Tatiana Anikina, and Tanja Baeumel. 2025. What Language(s) Does Aya-23 Think In? How Multilinguality Affects Internal Language Representations. In Proceedings of the Workshop on Beyond English: Natural Language Processing for all Languages in an Era of Large Language Models, pages 159–171, Varna, Bulgaria. INCOMA Ltd., Shoumen, BULGARIA.