SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models

Anil Ramakrishna; Yixin Wan; Xiaomeng Jin; Kai-Wei Chang; Zhiqi Bu; Bhanukiran Vinzamuri; Volkan Cevher; Mingyi Hong; Rahul Gupta

SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models

Anil Ramakrishna, Yixin Wan, Xiaomeng Jin, Kai-Wei Chang, Zhiqi Bu, Bhanukiran Vinzamuri, Volkan Cevher, Mingyi Hong, Rahul Gupta

Correct Metadata for

Important: The Anthology treat PDFs as authoritative. Please use this form only to correct data that is out of line with the PDF. See our corrections guidelines if you need to change the PDF.

Title Adjust the title. Retain tags such as <fixed-case>.

Authors Adjust author names and order to match the PDF.

Abstract Correct abstract if needed. Retain XML formatting tags such as <tex-math>.

Verification against PDF Ensure that the new title/authors match the snapshot below. (If there is no snapshot or it is too small, consult the PDF.)

Authors concatenated from the text boxes above:

ALL author names match the snapshot above—including middle initials, hyphens, and accents.

Abstract

We introduce SemEval-2025 Task 4: unlearn- ing sensitive content from Large Language Models (LLMs). The task features 3 subtasks for LLM unlearning spanning different use cases: (1) unlearn long form synthetic creative documents spanning different genres; (2) un- learn short form synthetic biographies contain- ing personally identifiable information (PII), in- cluding fake names, phone number, SSN, email and home addresses, and (3) unlearn real docu- ments sampled from the target model’s training dataset. We received over 100 submissions from over 30 institutions and we summarize the key techniques and lessons in this paper.

Anthology ID:: 2025.semeval-1.329
Volume:: Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
Month:: July
Year:: 2025
Address:: Vienna, Austria
Editors:: Sara Rosenthal, Aiala Rosá, Debanjan Ghosh, Marcos Zampieri
Venues:: SemEval | WS
SIG:
Publisher:: Association for Computational Linguistics
Note:
Pages:: 2584–2596
Language:
URL:: https://aclanthology.org/2025.semeval-1.329/
DOI:
Bibkey:
Cite (ACL):: Anil Ramakrishna, Yixin Wan, Xiaomeng Jin, Kai-Wei Chang, Zhiqi Bu, Bhanukiran Vinzamuri, Volkan Cevher, Mingyi Hong, and Rahul Gupta. 2025. SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 2584–2596, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):: SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models (Ramakrishna et al., SemEval 2025)
Copy Citation:
PDF:: https://aclanthology.org/2025.semeval-1.329.pdf

PDF Cite Search Fix data

Export citation

BibTeX
MODS XML
Endnote
Preformatted

@inproceedings{ramakrishna-etal-2025-semeval,
    title = "{S}em{E}val-2025 Task 4: Unlearning sensitive content from Large Language Models",
    author = "Ramakrishna, Anil  and
      Wan, Yixin  and
      Jin, Xiaomeng  and
      Chang, Kai-Wei  and
      Bu, Zhiqi  and
      Vinzamuri, Bhanukiran  and
      Cevher, Volkan  and
      Hong, Mingyi  and
      Gupta, Rahul",
    editor = "Rosenthal, Sara  and
      Ros{\'a}, Aiala  and
      Ghosh, Debanjan  and
      Zampieri, Marcos",
    booktitle = "Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.semeval-1.329/",
    pages = "2584--2596",
    ISBN = "979-8-89176-273-2",
    abstract = "We introduce SemEval-2025 Task 4: unlearn- ing sensitive content from Large Language Models (LLMs). The task features 3 subtasks for LLM unlearning spanning different use cases: (1) unlearn long form synthetic creative documents spanning different genres; (2) un- learn short form synthetic biographies contain- ing personally identifiable information (PII), in- cluding fake names, phone number, SSN, email and home addresses, and (3) unlearn real docu- ments sampled from the target model{'}s training dataset. We received over 100 submissions from over 30 institutions and we summarize the key techniques and lessons in this paper."
}

Download as File

<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="ramakrishna-etal-2025-semeval">
    <titleInfo>
        <title>SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Anil</namePart>
        <namePart type="family">Ramakrishna</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yixin</namePart>
        <namePart type="family">Wan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Xiaomeng</namePart>
        <namePart type="family">Jin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kai-Wei</namePart>
        <namePart type="family">Chang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Zhiqi</namePart>
        <namePart type="family">Bu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Bhanukiran</namePart>
        <namePart type="family">Vinzamuri</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Volkan</namePart>
        <namePart type="family">Cevher</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mingyi</namePart>
        <namePart type="family">Hong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Rahul</namePart>
        <namePart type="family">Gupta</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Sara</namePart>
            <namePart type="family">Rosenthal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Aiala</namePart>
            <namePart type="family">Rosá</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Debanjan</namePart>
            <namePart type="family">Ghosh</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marcos</namePart>
            <namePart type="family">Zampieri</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Vienna, Austria</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-273-2</identifier>
    </relatedItem>
    <abstract>We introduce SemEval-2025 Task 4: unlearn- ing sensitive content from Large Language Models (LLMs). The task features 3 subtasks for LLM unlearning spanning different use cases: (1) unlearn long form synthetic creative documents spanning different genres; (2) un- learn short form synthetic biographies contain- ing personally identifiable information (PII), in- cluding fake names, phone number, SSN, email and home addresses, and (3) unlearn real docu- ments sampled from the target model’s training dataset. We received over 100 submissions from over 30 institutions and we summarize the key techniques and lessons in this paper.</abstract>
    <identifier type="citekey">ramakrishna-etal-2025-semeval</identifier>
    <location>
        <url>https://aclanthology.org/2025.semeval-1.329/</url>
    </location>
    <part>
        <date>2025-07</date>
        <extent unit="page">
            <start>2584</start>
            <end>2596</end>
        </extent>
    </part>
</mods>
</modsCollection>

Download as File

%0 Conference Proceedings
%T SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models
%A Ramakrishna, Anil
%A Wan, Yixin
%A Jin, Xiaomeng
%A Chang, Kai-Wei
%A Bu, Zhiqi
%A Vinzamuri, Bhanukiran
%A Cevher, Volkan
%A Hong, Mingyi
%A Gupta, Rahul
%Y Rosenthal, Sara
%Y Rosá, Aiala
%Y Ghosh, Debanjan
%Y Zampieri, Marcos
%S Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
%D 2025
%8 July
%I Association for Computational Linguistics
%C Vienna, Austria
%@ 979-8-89176-273-2
%F ramakrishna-etal-2025-semeval
%X We introduce SemEval-2025 Task 4: unlearn- ing sensitive content from Large Language Models (LLMs). The task features 3 subtasks for LLM unlearning spanning different use cases: (1) unlearn long form synthetic creative documents spanning different genres; (2) un- learn short form synthetic biographies contain- ing personally identifiable information (PII), in- cluding fake names, phone number, SSN, email and home addresses, and (3) unlearn real docu- ments sampled from the target model’s training dataset. We received over 100 submissions from over 30 institutions and we summarize the key techniques and lessons in this paper.
%U https://aclanthology.org/2025.semeval-1.329/
%P 2584-2596

Download as File

Markdown (Informal)

[SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models](https://aclanthology.org/2025.semeval-1.329/) (Ramakrishna et al., SemEval 2025)

SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models (Ramakrishna et al., SemEval 2025)

ACL

Anil Ramakrishna, Yixin Wan, Xiaomeng Jin, Kai-Wei Chang, Zhiqi Bu, Bhanukiran Vinzamuri, Volkan Cevher, Mingyi Hong, and Rahul Gupta. 2025. SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 2584–2596, Vienna, Austria. Association for Computational Linguistics.