One Polluted Page Is Enough: Evaluating Web Content Pollution in LLM Recommenders

Minghao Luo, Liang Chen


Abstract
Search-augmented LLMs increasingly mediate everyday consumer recommendations by retrieving live web content. This creates a new risk: LLM recommenders may consume web content that Generative Engine Optimization (GEO) operators have polluted to mislead them. We ask: to what extent do they become unwitting promoters of fake products? We introduce FORGE (Fake Online Recommendations in Generative Environments), which locally rewrites real products in a frozen set of retrieved web pages into fake ones and measures how often the LLM recommends the fake product, across 225 real products in 15 categories and 5 consumer scenarios. Across 12 commercial and open-weights LLMs, all models are vulnerable: a single polluted page yields fooled rates of up to 27%, while the full top-3 replacement raises this to 73.8%. Vulnerability varies across categories, increasing when models lack stable prior knowledge of the products. Reasoning does not mitigate this vulnerability; instead, it often generates spurious social proof to justify false recommendations. None of the four defenses is adequate: the skepticism prompt can exacerbate vulnerability much like reasoning, the two consensus filters risk suppressing legitimate products, and credibility re-ranking helps every model but removes only a sixth of the fakes. We release the FORGE benchmark and the evaluation code at https://github.com/leoluolol/forge-benchmark.
Anthology ID:
2026.findings-emnlp.1815
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
38779–38805
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1815/
DOI:
Bibkey:
Cite (ACL):
Minghao Luo and Liang Chen. 2026. One Polluted Page Is Enough: Evaluating Web Content Pollution in LLM Recommenders. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 38779–38805, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
One Polluted Page Is Enough: Evaluating Web Content Pollution in LLM Recommenders (Luo & Chen, Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1815.pdf