<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="faisal-anastasopoulos-2024-efficient">
    <titleInfo>
        <title>An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Fahim</namePart>
        <namePart type="family">Faisal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Antonios</namePart>
        <namePart type="family">Anastasopoulos</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Fourth Workshop on Multilingual Representation Learning (MRL 2024)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Jonne</namePart>
            <namePart type="family">Sälevä</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Abraham</namePart>
            <namePart type="family">Owodunni</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The capacity and effectiveness of pre-trained multilingual models (MLMs) for zero-shot cross-lingual transfer is well established. However, phenomena of positive or negative transfer, and the effect of language choice still need to be fully understood, especially in the complex setting of massively multilingual LMs. We propose an efficient method to study transfer language influence in zero-shot performance on another target language. Unlike previous work, our approach disentangles downstream tasks from language, using dedicated adapter units. Our findings suggest that some languages do not largely affect others, while some languages, especially ones unseen during pre-training, can be extremely beneficial or detrimental for different target languages. We find that no transfer language is beneficial for all target languages. We do, curiously, observe languages previously unseen by MLMs consistently benefit from transfer from almost any language. We additionally use our modular approach to quantify negative interference efficiently and categorize languages accordingly. Furthermore, we provide a list of promising transfer-target language configurations that consistently lead to target language performance improvements.</abstract>
    <identifier type="citekey">faisal-anastasopoulos-2024-efficient</identifier>
    <location>
        <url>https://aclanthology.org/2024.mrl-1.4</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>45</start>
            <end>92</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Models
%A Faisal, Fahim
%A Anastasopoulos, Antonios
%Y Sälevä, Jonne
%Y Owodunni, Abraham
%S Proceedings of the Fourth Workshop on Multilingual Representation Learning (MRL 2024)
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F faisal-anastasopoulos-2024-efficient
%X The capacity and effectiveness of pre-trained multilingual models (MLMs) for zero-shot cross-lingual transfer is well established. However, phenomena of positive or negative transfer, and the effect of language choice still need to be fully understood, especially in the complex setting of massively multilingual LMs. We propose an efficient method to study transfer language influence in zero-shot performance on another target language. Unlike previous work, our approach disentangles downstream tasks from language, using dedicated adapter units. Our findings suggest that some languages do not largely affect others, while some languages, especially ones unseen during pre-training, can be extremely beneficial or detrimental for different target languages. We find that no transfer language is beneficial for all target languages. We do, curiously, observe languages previously unseen by MLMs consistently benefit from transfer from almost any language. We additionally use our modular approach to quantify negative interference efficiently and categorize languages accordingly. Furthermore, we provide a list of promising transfer-target language configurations that consistently lead to target language performance improvements.
%U https://aclanthology.org/2024.mrl-1.4
%P 45-92

Markdown (Informal)

[An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Models](https://aclanthology.org/2024.mrl-1.4) (Faisal & Anastasopoulos, MRL 2024)

• An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Models (Faisal & Anastasopoulos, MRL 2024)

ACL

• Fahim Faisal and Antonios Anastasopoulos. 2024. An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Models. In Proceedings of the Fourth Workshop on Multilingual Representation Learning (MRL 2024), pages 45–92, Miami, Florida, USA. Association for Computational Linguistics.