<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="li-chu-2024-continually">
    <titleInfo>
        <title>Can We Continually Edit Language Models? On the Knowledge Attenuation in Sequential Model Editing</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Qi</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Xiaowen</namePart>
        <namePart type="family">Chu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics ACL 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Lun-Wei</namePart>
            <namePart type="family">Ku</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andre</namePart>
            <namePart type="family">Martins</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vivek</namePart>
            <namePart type="family">Srikumar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand and virtual meeting</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Model editing has become a promising method for precisely and effectively updating knowledge in language models. In this paper, we investigate knowledge attenuation, in which the retention of updated knowledge within the language model decreases as the number of edits increases after sequential editing. Through empirical study, we discovered that existing editing methods generally suffer from knowledge attenuation. We attribute this phenomenon to two aspects: (1) redundant parameters interference and (2) update weight disentanglement. To this end, we propose the AdaPLE method. It not only mitigates the knowledge attenuation issue but also improves the performance on existing benchmarks. To the best of our knowledge, we are the first to investigate the cause and mitigation of knowledge attenuation in sequential LLM editing.</abstract>
    <identifier type="citekey">li-chu-2024-continually</identifier>
    <location>
        <url>https://aclanthology.org/2024.findings-acl.323</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>5438</start>
            <end>5455</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Can We Continually Edit Language Models? On the Knowledge Attenuation in Sequential Model Editing
%A Li, Qi
%A Chu, Xiaowen
%Y Ku, Lun-Wei
%Y Martins, Andre
%Y Srikumar, Vivek
%S Findings of the Association for Computational Linguistics ACL 2024
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand and virtual meeting
%F li-chu-2024-continually
%X Model editing has become a promising method for precisely and effectively updating knowledge in language models. In this paper, we investigate knowledge attenuation, in which the retention of updated knowledge within the language model decreases as the number of edits increases after sequential editing. Through empirical study, we discovered that existing editing methods generally suffer from knowledge attenuation. We attribute this phenomenon to two aspects: (1) redundant parameters interference and (2) update weight disentanglement. To this end, we propose the AdaPLE method. It not only mitigates the knowledge attenuation issue but also improves the performance on existing benchmarks. To the best of our knowledge, we are the first to investigate the cause and mitigation of knowledge attenuation in sequential LLM editing.
%U https://aclanthology.org/2024.findings-acl.323
%P 5438-5455

Markdown (Informal)

[Can We Continually Edit Language Models? On the Knowledge Attenuation in Sequential Model Editing](https://aclanthology.org/2024.findings-acl.323) (Li & Chu, Findings 2024)

• Can We Continually Edit Language Models? On the Knowledge Attenuation in Sequential Model Editing (Li & Chu, Findings 2024)

ACL

• Qi Li and Xiaowen Chu. 2024. Can We Continually Edit Language Models? On the Knowledge Attenuation in Sequential Model Editing. In Findings of the Association for Computational Linguistics ACL 2024, pages 5438–5455, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.