<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="qin-etal-2023-lifelong">
    <titleInfo>
        <title>Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Chengwei</namePart>
        <namePart type="family">Qin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chen</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Shafiq</namePart>
        <namePart type="family">Joty</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Lifelong sequence generation (LSG), a problem in continual learning, aims to continually train a model on a sequence of generation tasks to learn constantly emerging new generation patterns while avoiding the forgetting of previous knowledge. Existing LSG methods mainly focus on maintaining old knowledge while paying little attention to knowledge transfer across tasks. In contrast, humans can better learn new tasks by leveraging previously acquired knowledge from similar tasks. Inspired by the learning paradigm of humans, we propose Dynamic Module Expansion and Adaptation (DMEA), which enables the model to dynamically determine the architecture for acquiring new knowledge based on task correlation and select the most similar previous tasks to facilitate adaptation to new tasks. In addition, as the learning process can easily be biased towards the current task which might cause more severe forgetting of previously learned knowledge, we propose dynamic gradient scaling to balance the learning of the current task and replayed tasks. With extensive experiments, we demonstrate that DMEA can consistently outperform existing methods in different LSG settings.</abstract>
    <identifier type="citekey">qin-etal-2023-lifelong</identifier>
    <identifier type="doi">10.18653/v1/2023.emnlp-main.414</identifier>
    <location>
        <url>https://aclanthology.org/2023.emnlp-main.414</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>6701</start>
            <end>6714</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation
%A Qin, Chengwei
%A Chen, Chen
%A Joty, Shafiq
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F qin-etal-2023-lifelong
%X Lifelong sequence generation (LSG), a problem in continual learning, aims to continually train a model on a sequence of generation tasks to learn constantly emerging new generation patterns while avoiding the forgetting of previous knowledge. Existing LSG methods mainly focus on maintaining old knowledge while paying little attention to knowledge transfer across tasks. In contrast, humans can better learn new tasks by leveraging previously acquired knowledge from similar tasks. Inspired by the learning paradigm of humans, we propose Dynamic Module Expansion and Adaptation (DMEA), which enables the model to dynamically determine the architecture for acquiring new knowledge based on task correlation and select the most similar previous tasks to facilitate adaptation to new tasks. In addition, as the learning process can easily be biased towards the current task which might cause more severe forgetting of previously learned knowledge, we propose dynamic gradient scaling to balance the learning of the current task and replayed tasks. With extensive experiments, we demonstrate that DMEA can consistently outperform existing methods in different LSG settings.
%R 10.18653/v1/2023.emnlp-main.414
%U https://aclanthology.org/2023.emnlp-main.414
%U https://doi.org/10.18653/v1/2023.emnlp-main.414
%P 6701-6714

Markdown (Informal)

[Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation](https://aclanthology.org/2023.emnlp-main.414) (Qin et al., EMNLP 2023)

• Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation (Qin et al., EMNLP 2023)

ACL

• Chengwei Qin, Chen Chen, and Shafiq Joty. 2023. Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6701–6714, Singapore. Association for Computational Linguistics.