<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kanwatchara-etal-2022-enhancing">
    <titleInfo>
        <title>Enhancing Lifelong Language Learning by Improving Pseudo-Sample Generation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Kasidis</namePart>
        <namePart type="family">Kanwatchara</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Thanapapas</namePart>
        <namePart type="family">Horsuwan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Piyawat</namePart>
        <namePart type="family">Lertvittayakumjorn</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Boonserm</namePart>
        <namePart type="family">Kijsirikul</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Peerapon</namePart>
        <namePart type="family">Vateekul</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>To achieve lifelong language learning, pseudo-rehearsal methods leverage samples generated from a language model to refresh the knowledge of previously learned tasks. Without proper controls, however, these methods could fail to retain the knowledge of complex tasks with longer texts since most of the generated samples are low in quality. To overcome the problem, we propose three specific contributions. First, we utilize double language models, each of which specializes in a specific part of the input, to produce high-quality pseudo samples. Second, we reduce the number of parameters used by applying adapter modules to enhance training efficiency. Third, we further improve the overall quality of pseudo samples using temporal ensembling and sample regeneration. The results show that our framework achieves significant improvement over baselines on multiple task sequences. Also, our pseudo sample analysis reveals helpful insights for designing even better pseudo-rehearsal methods in the future.</abstract>
    <identifier type="citekey">kanwatchara-etal-2022-enhancing</identifier>
    <identifier type="doi">10.1162/coli_a_00449</identifier>
    <location>
        <url>https://aclanthology.org/2022.cl-4.12</url>
    </location>
    <part>
        <date>2022-12</date>
        <detail type="volume"><number>48</number></detail>
        <detail type="issue"><number>4</number></detail>
        <extent unit="page">
            <start>819</start>
            <end>848</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T Enhancing Lifelong Language Learning by Improving Pseudo-Sample Generation
%A Kanwatchara, Kasidis
%A Horsuwan, Thanapapas
%A Lertvittayakumjorn, Piyawat
%A Kijsirikul, Boonserm
%A Vateekul, Peerapon
%J Computational Linguistics
%D 2022
%8 December
%V 48
%N 4
%I MIT Press
%C Cambridge, MA
%F kanwatchara-etal-2022-enhancing
%X To achieve lifelong language learning, pseudo-rehearsal methods leverage samples generated from a language model to refresh the knowledge of previously learned tasks. Without proper controls, however, these methods could fail to retain the knowledge of complex tasks with longer texts since most of the generated samples are low in quality. To overcome the problem, we propose three specific contributions. First, we utilize double language models, each of which specializes in a specific part of the input, to produce high-quality pseudo samples. Second, we reduce the number of parameters used by applying adapter modules to enhance training efficiency. Third, we further improve the overall quality of pseudo samples using temporal ensembling and sample regeneration. The results show that our framework achieves significant improvement over baselines on multiple task sequences. Also, our pseudo sample analysis reveals helpful insights for designing even better pseudo-rehearsal methods in the future.
%R 10.1162/coli_a_00449
%U https://aclanthology.org/2022.cl-4.12
%U https://doi.org/10.1162/coli_a_00449
%P 819-848

Markdown (Informal)

[Enhancing Lifelong Language Learning by Improving Pseudo-Sample Generation](https://aclanthology.org/2022.cl-4.12) (Kanwatchara et al., CL 2022)

• Enhancing Lifelong Language Learning by Improving Pseudo-Sample Generation (Kanwatchara et al., CL 2022)

ACL

• Kasidis Kanwatchara, Thanapapas Horsuwan, Piyawat Lertvittayakumjorn, Boonserm Kijsirikul, and Peerapon Vateekul. 2022. Enhancing Lifelong Language Learning by Improving Pseudo-Sample Generation. Computational Linguistics, 48(4):819–848.