2026

Online scams are a growing form of digital harm, yet scam research remains constrained by fragmented datasets and limited annotation depth. We introduce SCAMBENCH, a benchmark of 1,836 authentic scam messages curated from victim-reported incidents across six scam types: employment, investment, imposter, sweepstakes/lottery/prizes, tech support, and advance fee loan scams, along with 1,834 real-world non-scam examples from legitimate sources. SCAMBENCH provides multi-dimensional annotations that capture persuasion strategies from the scammer perspective, red flags from the user perspective, and severity levels reflecting potential harm and urgency. These annotations are produced through a human-AI collaborative pipeline that uses expert-labeled examples to calibrate and validate LLM-assisted annotation at scale. Our analysis shows that these signals vary significantly across scam types, highlighting substantial scam-type heterogeneity. We further use annotated evidence spans to generate cue-removed hard scam examples by rephrasing explicit persuasion and red-flag cues. Evaluating five LLM-based detectors on original and rewritten scams shows that all five detectors perform worse on cue-removed examples, and the performance drop persists on a human-validated subset, suggesting sensitivity to surface-level cues. A qualitative case study identifies five recurring failure patterns. SCAMBENCH provides a resource for scam detection, robustness evaluation, and scam education, publicly available at https://huggingface.co/datasets/Shouninger/ScamBench.