LongPIBench: A Long-Context Benchmark for Prompt Injection

Yupei Liu, Yuqi Jia, Neil Zhenqiang Gong, Jinyuan Jia


Abstract
Prompt injection attacks pose a serious security risk to large language models in real-world applications. However, existing prompt injection benchmarks primarily focus on short-context inputs, leaving the attacks and defenses in long-context settings largely unexplored. This gap leads to a substantial overestimation of the effectiveness of current defenses. In this paper, we bridge the gap by introducing LongPIBench, a long-context benchmark for prompt injection covering 4 realistic application scenarios: paper peer review, resume screening, code review, and email summary. For each scenario, we construct a synthetic dataset and a real-world dataset, with context lengths ranging from thousands to tens of thousands of tokens. The evaluation results on LongPIBench reveal significant vulnerabilities of prompt injection defenses under long-context settings: even simple heuristic prompt injection attacks achieve high success rates and frequently bypass state-of-the-art defenses. We hope LongPIBench can serve as a practical benchmark for systematically evaluating prompt injection defenses in realistic long-context scenarios.
Anthology ID:
2026.findings-emnlp.1510
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
32132–32149
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1510/
DOI:
Bibkey:
Cite (ACL):
Yupei Liu, Yuqi Jia, Neil Zhenqiang Gong, and Jinyuan Jia. 2026. LongPIBench: A Long-Context Benchmark for Prompt Injection. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 32132–32149, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
LongPIBench: A Long-Context Benchmark for Prompt Injection (Liu et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1510.pdf