<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="carrion-casacuberta-2022-shot">
    <titleInfo>
        <title>Few-Shot Regularization to Tackle Catastrophic Forgetting in Multilingual Machine Translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Salvador</namePart>
        <namePart type="family">Carrión</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Francisco</namePart>
        <namePart type="family">Casacuberta</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 15th biennial conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Kevin</namePart>
            <namePart type="family">Duh</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Francisco</namePart>
            <namePart type="family">Guzmán</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Machine Translation in the Americas</publisher>
            <place>
                <placeTerm type="text">Orlando, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Increasing the number of tasks supported by a machine learning model without forgetting previously learned tasks is the goal of any lifelong learning system. In this work, we study how to mitigate the effects of the catastrophic forgetting problem to sequentially train a multilingual neural machine translation model using minimal past information. First, we describe the catastrophic forgetting phenomenon as a function of the number of tasks learned (language pairs) and the ratios of past data used during the learning of the new task. Next, we explore the importance of applying oversampling strategies for scenarios where only minimal amounts of past data are available. Finally, we derive a new loss function that minimizes the forgetting of previously learned tasks by actively re-weighting past samples and penalizing weights that deviate too much from the original model. Our work suggests that by using minimal amounts of past data and a simple regularization function, we can significantly mitigate the effects of the catastrophic forgetting phenomenon without increasing the computational costs.</abstract>
    <identifier type="citekey">carrion-casacuberta-2022-shot</identifier>
    <location>
        <url>https://aclanthology.org/2022.amta-research.14</url>
    </location>
    <part>
        <date>2022-09</date>
        <extent unit="page">
            <start>188</start>
            <end>199</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Few-Shot Regularization to Tackle Catastrophic Forgetting in Multilingual Machine Translation
%A Carrión, Salvador
%A Casacuberta, Francisco
%Y Duh, Kevin
%Y Guzmán, Francisco
%S Proceedings of the 15th biennial conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)
%D 2022
%8 September
%I Association for Machine Translation in the Americas
%C Orlando, USA
%F carrion-casacuberta-2022-shot
%X Increasing the number of tasks supported by a machine learning model without forgetting previously learned tasks is the goal of any lifelong learning system. In this work, we study how to mitigate the effects of the catastrophic forgetting problem to sequentially train a multilingual neural machine translation model using minimal past information. First, we describe the catastrophic forgetting phenomenon as a function of the number of tasks learned (language pairs) and the ratios of past data used during the learning of the new task. Next, we explore the importance of applying oversampling strategies for scenarios where only minimal amounts of past data are available. Finally, we derive a new loss function that minimizes the forgetting of previously learned tasks by actively re-weighting past samples and penalizing weights that deviate too much from the original model. Our work suggests that by using minimal amounts of past data and a simple regularization function, we can significantly mitigate the effects of the catastrophic forgetting phenomenon without increasing the computational costs.
%U https://aclanthology.org/2022.amta-research.14
%P 188-199

Markdown (Informal)

[Few-Shot Regularization to Tackle Catastrophic Forgetting in Multilingual Machine Translation](https://aclanthology.org/2022.amta-research.14) (Carrión & Casacuberta, AMTA 2022)

• Few-Shot Regularization to Tackle Catastrophic Forgetting in Multilingual Machine Translation (Carrión & Casacuberta, AMTA 2022)

ACL

• Salvador Carrión and Francisco Casacuberta. 2022. Few-Shot Regularization to Tackle Catastrophic Forgetting in Multilingual Machine Translation. In Proceedings of the 15th biennial conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), pages 188–199, Orlando, USA. Association for Machine Translation in the Americas.