<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="li-murray-2023-zero">
    <titleInfo>
        <title>Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Tianjian</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kenton</namePart>
        <namePart type="family">Murray</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jordan</namePart>
            <namePart type="family">Boyd-Graber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Naoaki</namePart>
            <namePart type="family">Okazaki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Zero-shot cross-lingual transfer is when a multilingual model is trained to perform a task in one language and then is applied to another language. Although the zero-shot cross-lingual transfer approach has achieved success in various classification tasks, its performance on natural language generation tasks falls short in quality and sometimes outputs an incorrect language. In our study, we show that the fine-tuning process learns language invariant representations, which is beneficial for classification tasks but harmful for generation tasks. Motivated by this, we propose a simple method to regularize the model from learning language invariant representations and a method to select model checkpoints without a development set in the target language, both resulting in better generation quality. Experiments on three semantically diverse generation tasks show that our method reduces the accidental translation problem by 68% and improves the ROUGE-L score by 1.5 on average.</abstract>
    <identifier type="citekey">li-murray-2023-zero</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-acl.789</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-acl.789</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>12461</start>
            <end>12476</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution
%A Li, Tianjian
%A Murray, Kenton
%Y Rogers, Anna
%Y Boyd-Graber, Jordan
%Y Okazaki, Naoaki
%S Findings of the Association for Computational Linguistics: ACL 2023
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F li-murray-2023-zero
%X Zero-shot cross-lingual transfer is when a multilingual model is trained to perform a task in one language and then is applied to another language. Although the zero-shot cross-lingual transfer approach has achieved success in various classification tasks, its performance on natural language generation tasks falls short in quality and sometimes outputs an incorrect language. In our study, we show that the fine-tuning process learns language invariant representations, which is beneficial for classification tasks but harmful for generation tasks. Motivated by this, we propose a simple method to regularize the model from learning language invariant representations and a method to select model checkpoints without a development set in the target language, both resulting in better generation quality. Experiments on three semantically diverse generation tasks show that our method reduces the accidental translation problem by 68% and improves the ROUGE-L score by 1.5 on average.
%R 10.18653/v1/2023.findings-acl.789
%U https://aclanthology.org/2023.findings-acl.789
%U https://doi.org/10.18653/v1/2023.findings-acl.789
%P 12461-12476

Markdown (Informal)

[Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution](https://aclanthology.org/2023.findings-acl.789) (Li & Murray, Findings 2023)

• Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution (Li & Murray, Findings 2023)

ACL

• Tianjian Li and Kenton Murray. 2023. Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution. In Findings of the Association for Computational Linguistics: ACL 2023, pages 12461–12476, Toronto, Canada. Association for Computational Linguistics.