<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="chi-etal-2023-cross">
    <titleInfo>
        <title>Can Cross-Lingual Transferability of Multilingual Transformers Be Activated Without End-Task Data?</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Zewen</namePart>
        <namePart type="family">Chi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Heyan</namePart>
        <namePart type="family">Huang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Xian-Ling</namePart>
        <namePart type="family">Mao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jordan</namePart>
            <namePart type="family">Boyd-Graber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Naoaki</namePart>
            <namePart type="family">Okazaki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Pretrained multilingual Transformers have achieved great success in cross-lingual transfer learning. Current methods typically activate the cross-lingual transferability of multilingual Transformers by fine-tuning them on end-task data. However, the methods cannot perform cross-lingual transfer when end-task data are unavailable. In this work, we explore whether the cross-lingual transferability can be activated without end-task data. We propose a cross-lingual transfer method, named PlugIn-X. PlugIn-X disassembles monolingual and multilingual Transformers into sub-modules, and reassembles them to be the multilingual end-task model. After representation adaptation, PlugIn-X finally performs cross-lingual transfer in a plug-and-play style. Experimental results show that PlugIn-X successfully activates the cross-lingual transferability of multilingual Transformers without accessing end-task data. Moreover, we analyze how the cross-model representation alignment affects the cross-lingual transferability.</abstract>
    <identifier type="citekey">chi-etal-2023-cross</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-acl.796</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-acl.796</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>12572</start>
            <end>12584</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Can Cross-Lingual Transferability of Multilingual Transformers Be Activated Without End-Task Data?
%A Chi, Zewen
%A Huang, Heyan
%A Mao, Xian-Ling
%Y Rogers, Anna
%Y Boyd-Graber, Jordan
%Y Okazaki, Naoaki
%S Findings of the Association for Computational Linguistics: ACL 2023
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F chi-etal-2023-cross
%X Pretrained multilingual Transformers have achieved great success in cross-lingual transfer learning. Current methods typically activate the cross-lingual transferability of multilingual Transformers by fine-tuning them on end-task data. However, the methods cannot perform cross-lingual transfer when end-task data are unavailable. In this work, we explore whether the cross-lingual transferability can be activated without end-task data. We propose a cross-lingual transfer method, named PlugIn-X. PlugIn-X disassembles monolingual and multilingual Transformers into sub-modules, and reassembles them to be the multilingual end-task model. After representation adaptation, PlugIn-X finally performs cross-lingual transfer in a plug-and-play style. Experimental results show that PlugIn-X successfully activates the cross-lingual transferability of multilingual Transformers without accessing end-task data. Moreover, we analyze how the cross-model representation alignment affects the cross-lingual transferability.
%R 10.18653/v1/2023.findings-acl.796
%U https://aclanthology.org/2023.findings-acl.796
%U https://doi.org/10.18653/v1/2023.findings-acl.796
%P 12572-12584

Markdown (Informal)

[Can Cross-Lingual Transferability of Multilingual Transformers Be Activated Without End-Task Data?](https://aclanthology.org/2023.findings-acl.796) (Chi et al., Findings 2023)

• Can Cross-Lingual Transferability of Multilingual Transformers Be Activated Without End-Task Data? (Chi et al., Findings 2023)

ACL

• Zewen Chi, Heyan Huang, and Xian-Ling Mao. 2023. Can Cross-Lingual Transferability of Multilingual Transformers Be Activated Without End-Task Data?. In Findings of the Association for Computational Linguistics: ACL 2023, pages 12572–12584, Toronto, Canada. Association for Computational Linguistics.