<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lizzo-heck-2025-unlearn">
    <titleInfo>
        <title>UNLEARN Efficient Removal of Knowledge in Large Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Tyler</namePart>
        <namePart type="family">Lizzo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Larry</namePart>
        <namePart type="family">Heck</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-04</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: NAACL 2025</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Luis</namePart>
            <namePart type="family">Chiruzzo</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Alan</namePart>
            <namePart type="family">Ritter</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lu</namePart>
            <namePart type="family">Wang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Albuquerque, New Mexico</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-195-7</identifier>
    </relatedItem>
    <abstract>Large Language Models (LLMs) excel in many Natural Language Processing tasks but are outperformed by specialized tools for certain tasks. This raises the question: Can we reduce redundant LLM parameters when using these tools? Given the size and high training costs of LLMs, it is essential to efficiently forget specific knowledge without retraining. This paper introduces UNLEARN, a novel method that uses subspace techniques to selectively remove knowledge without access to the original training data, without retraining, and with minimal impact to other tasks. Our results show that UNLEARN significantly outperforms previous methods for forgetting targeted (unwanted) knowledge while also preserving related (wanted) knowledge. We also propose LEARN, a complementary approach for targeted knowledge addition, which achieves fine-tuning accuracy comparable to Low-Rank Adaptation (LoRA) without degrading related task performance.</abstract>
    <identifier type="citekey">lizzo-heck-2025-unlearn</identifier>
    <identifier type="doi">10.18653/v1/2025.findings-naacl.405</identifier>
    <location>
        <url>https://aclanthology.org/2025.findings-naacl.405/</url>
    </location>
    <part>
        <date>2025-04</date>
        <extent unit="page">
            <start>7257</start>
            <end>7268</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T UNLEARN Efficient Removal of Knowledge in Large Language Models
%A Lizzo, Tyler
%A Heck, Larry
%Y Chiruzzo, Luis
%Y Ritter, Alan
%Y Wang, Lu
%S Findings of the Association for Computational Linguistics: NAACL 2025
%D 2025
%8 April
%I Association for Computational Linguistics
%C Albuquerque, New Mexico
%@ 979-8-89176-195-7
%F lizzo-heck-2025-unlearn
%X Large Language Models (LLMs) excel in many Natural Language Processing tasks but are outperformed by specialized tools for certain tasks. This raises the question: Can we reduce redundant LLM parameters when using these tools? Given the size and high training costs of LLMs, it is essential to efficiently forget specific knowledge without retraining. This paper introduces UNLEARN, a novel method that uses subspace techniques to selectively remove knowledge without access to the original training data, without retraining, and with minimal impact to other tasks. Our results show that UNLEARN significantly outperforms previous methods for forgetting targeted (unwanted) knowledge while also preserving related (wanted) knowledge. We also propose LEARN, a complementary approach for targeted knowledge addition, which achieves fine-tuning accuracy comparable to Low-Rank Adaptation (LoRA) without degrading related task performance.
%R 10.18653/v1/2025.findings-naacl.405
%U https://aclanthology.org/2025.findings-naacl.405/
%U https://doi.org/10.18653/v1/2025.findings-naacl.405
%P 7257-7268

Markdown (Informal)

[UNLEARN Efficient Removal of Knowledge in Large Language Models](https://aclanthology.org/2025.findings-naacl.405/) (Lizzo & Heck, Findings 2025)

• UNLEARN Efficient Removal of Knowledge in Large Language Models (Lizzo & Heck, Findings 2025)

ACL

• Tyler Lizzo and Larry Heck. 2025. UNLEARN Efficient Removal of Knowledge in Large Language Models. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 7257–7268, Albuquerque, New Mexico. Association for Computational Linguistics.