Context-Aware Membership Inference Attacks against Pre-trained Large Language Models

Hongyan Chang; Ali Shahin Shamsabadi; Kleomenis Katevas; Hamed Haddadi; Reza Shokri

doi:10.18653/v1/2025.emnlp-main.370

Context-Aware Membership Inference Attacks against Pre-trained Large Language Models

Hongyan Chang, Ali Shahin Shamsabadi, Kleomenis Katevas, Hamed Haddadi, Reza Shokri

Correct Metadata for

Use this form to create a GitHub issue with structured data describing the correction. You will need a GitHub account. Once you create that issue, the correction will be reviewed by a staff member.

⚠️ Mobile Users: Submitting this form to create a new issue will only work with github.com, not the GitHub Mobile app.

Important: The Anthology treat PDFs as authoritative. Please use this form only to correct data that is out of line with the PDF. See our corrections guidelines if you need to change the PDF.

Title Adjust the title. Retain tags such as <fixed-case>.

Authors Adjust author names and order to match the PDF.

Abstract Correct abstract if needed. Retain XML formatting tags such as <tex-math>. You may use <b>...</b> for bold, <i>...</i> for italic, and <url>...</url> for URLs.

Verification against PDF Ensure that the new title/authors match the snapshot below. (If there is no snapshot or it is too small, consult the PDF.)

Authors concatenated from the text boxes above:

ALL author names match the snapshot above—including middle initials, hyphens, and accents.

Abstract

Membership Inference Attacks (MIAs) on pre-trained Large Language Models (LLMs) aim at determining if a data point was part of the model’s training set. Prior MIAs that are built for classification models fail at LLMs, due to ignoring the generative nature of LLMs across token sequences. In this paper, we present a novel attack on pre-trained LLMs that adapts MIA statistical tests to the perplexity dynamics of subsequences within a data point. Our method significantly outperforms prior approaches, revealing context-dependent memorization patterns in pre-trained LLMs.

Anthology ID:: 2025.emnlp-main.370
Volume:: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Month:: November
Year:: 2025
Address:: Suzhou, China
Editors:: Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:: EMNLP
SIG:
Publisher:: Association for Computational Linguistics
Note:
Pages:: 7288–7310
Language:
URL:: https://aclanthology.org/2025.emnlp-main.370/
DOI:: 10.18653/v1/2025.emnlp-main.370
Bibkey:
Cite (ACL):: Hongyan Chang, Ali Shahin Shamsabadi, Kleomenis Katevas, Hamed Haddadi, and Reza Shokri. 2025. Context-Aware Membership Inference Attacks against Pre-trained Large Language Models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 7288–7310, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):: Context-Aware Membership Inference Attacks against Pre-trained Large Language Models (Chang et al., EMNLP 2025)
Copy Citation:
PDF:: https://aclanthology.org/2025.emnlp-main.370.pdf
Checklist:: 2025.emnlp-main.370.checklist.pdf

PDF Cite Search Checklist Fix data

Export citation

BibTeX
MODS XML
Endnote
Preformatted

@inproceedings{chang-etal-2025-context,
    title = "Context-Aware Membership Inference Attacks against Pre-trained Large Language Models",
    author = "Chang, Hongyan  and
      Shahin Shamsabadi, Ali  and
      Katevas, Kleomenis  and
      Haddadi, Hamed  and
      Shokri, Reza",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.emnlp-main.370/",
    doi = "10.18653/v1/2025.emnlp-main.370",
    pages = "7288--7310",
    ISBN = "979-8-89176-332-6",
    abstract = "Membership Inference Attacks (MIAs) on pre-trained Large Language Models (LLMs) aim at determining if a data point was part of the model{'}s training set. Prior MIAs that are built for classification models fail at LLMs, due to ignoring the generative nature of LLMs across token sequences. In this paper, we present a novel attack on pre-trained LLMs that adapts MIA statistical tests to the perplexity dynamics of subsequences within a data point. Our method significantly outperforms prior approaches, revealing context-dependent memorization patterns in pre-trained LLMs."
}

Download as File

<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="chang-etal-2025-context">
    <titleInfo>
        <title>Context-Aware Membership Inference Attacks against Pre-trained Large Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hongyan</namePart>
        <namePart type="family">Chang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ali</namePart>
        <namePart type="family">Shahin Shamsabadi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kleomenis</namePart>
        <namePart type="family">Katevas</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hamed</namePart>
        <namePart type="family">Haddadi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Reza</namePart>
        <namePart type="family">Shokri</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Christos</namePart>
            <namePart type="family">Christodoulopoulos</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tanmoy</namePart>
            <namePart type="family">Chakraborty</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Carolyn</namePart>
            <namePart type="family">Rose</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Violet</namePart>
            <namePart type="family">Peng</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Suzhou, China</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-332-6</identifier>
    </relatedItem>
    <abstract>Membership Inference Attacks (MIAs) on pre-trained Large Language Models (LLMs) aim at determining if a data point was part of the model’s training set. Prior MIAs that are built for classification models fail at LLMs, due to ignoring the generative nature of LLMs across token sequences. In this paper, we present a novel attack on pre-trained LLMs that adapts MIA statistical tests to the perplexity dynamics of subsequences within a data point. Our method significantly outperforms prior approaches, revealing context-dependent memorization patterns in pre-trained LLMs.</abstract>
    <identifier type="citekey">chang-etal-2025-context</identifier>
    <identifier type="doi">10.18653/v1/2025.emnlp-main.370</identifier>
    <location>
        <url>https://aclanthology.org/2025.emnlp-main.370/</url>
    </location>
    <part>
        <date>2025-11</date>
        <extent unit="page">
            <start>7288</start>
            <end>7310</end>
        </extent>
    </part>
</mods>
</modsCollection>

Download as File

%0 Conference Proceedings
%T Context-Aware Membership Inference Attacks against Pre-trained Large Language Models
%A Chang, Hongyan
%A Shahin Shamsabadi, Ali
%A Katevas, Kleomenis
%A Haddadi, Hamed
%A Shokri, Reza
%Y Christodoulopoulos, Christos
%Y Chakraborty, Tanmoy
%Y Rose, Carolyn
%Y Peng, Violet
%S Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
%D 2025
%8 November
%I Association for Computational Linguistics
%C Suzhou, China
%@ 979-8-89176-332-6
%F chang-etal-2025-context
%X Membership Inference Attacks (MIAs) on pre-trained Large Language Models (LLMs) aim at determining if a data point was part of the model’s training set. Prior MIAs that are built for classification models fail at LLMs, due to ignoring the generative nature of LLMs across token sequences. In this paper, we present a novel attack on pre-trained LLMs that adapts MIA statistical tests to the perplexity dynamics of subsequences within a data point. Our method significantly outperforms prior approaches, revealing context-dependent memorization patterns in pre-trained LLMs.
%R 10.18653/v1/2025.emnlp-main.370
%U https://aclanthology.org/2025.emnlp-main.370/
%U https://doi.org/10.18653/v1/2025.emnlp-main.370
%P 7288-7310

Download as File

Markdown (Informal)

[Context-Aware Membership Inference Attacks against Pre-trained Large Language Models](https://aclanthology.org/2025.emnlp-main.370/) (Chang et al., EMNLP 2025)

Context-Aware Membership Inference Attacks against Pre-trained Large Language Models (Chang et al., EMNLP 2025)

ACL

Hongyan Chang, Ali Shahin Shamsabadi, Kleomenis Katevas, Hamed Haddadi, and Reza Shokri. 2025. Context-Aware Membership Inference Attacks against Pre-trained Large Language Models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 7288–7310, Suzhou, China. Association for Computational Linguistics.