<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="li-etal-2026-scambench">
    <titleInfo>
        <title>SCAMBENCH: A Multi-Dimensional Scam Benchmark through Human-AI Collaborative Data Construction</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Huaye</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Siyu</namePart>
        <namePart type="family">Huang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jinkyung</namePart>
        <namePart type="given">Katie</namePart>
        <namePart type="family">Park</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-462-0</identifier>
    </relatedItem>
    <abstract>Online scams are a growing form of digital harm, yet scam research remains constrained by fragmented datasets and limited annotation depth. We introduce SCAMBENCH, a benchmark of 1,836 authentic scam messages curated from victim-reported incidents across six scam types: employment, investment, imposter, sweepstakes/lottery/prizes, tech support, and advance fee loan scams, along with 1,834 real-world non-scam examples from legitimate sources. SCAMBENCH provides multi-dimensional annotations that capture persuasion strategies from the scammer perspective, red flags from the user perspective, and severity levels reflecting potential harm and urgency. These annotations are produced through a human-AI collaborative pipeline that uses expert-labeled examples to calibrate and validate LLM-assisted annotation at scale. Our analysis shows that these signals vary significantly across scam types, highlighting substantial scam-type heterogeneity. We further use annotated evidence spans to generate cue-removed hard scam examples by rephrasing explicit persuasion and red-flag cues. Evaluating five LLM-based detectors on original and rewritten scams shows that all five detectors perform worse on cue-removed examples, and the performance drop persists on a human-validated subset, suggesting sensitivity to surface-level cues. A qualitative case study identifies five recurring failure patterns. SCAMBENCH provides a resource for scam detection, robustness evaluation, and scam education, publicly available at https://huggingface.co/datasets/Shouninger/ScamBench.</abstract>
    <identifier type="citekey">li-etal-2026-scambench</identifier>
    <location>
        <url>https://aclanthology.org/2026.findings-emnlp.404/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>8540</start>
            <end>8563</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T SCAMBENCH: A Multi-Dimensional Scam Benchmark through Human-AI Collaborative Data Construction
%A Li, Huaye
%A Huang, Siyu
%A Park, Jinkyung Katie
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Findings of the Association for Computational Linguistics: EMNLP 2026
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-462-0
%F li-etal-2026-scambench
%X Online scams are a growing form of digital harm, yet scam research remains constrained by fragmented datasets and limited annotation depth. We introduce SCAMBENCH, a benchmark of 1,836 authentic scam messages curated from victim-reported incidents across six scam types: employment, investment, imposter, sweepstakes/lottery/prizes, tech support, and advance fee loan scams, along with 1,834 real-world non-scam examples from legitimate sources. SCAMBENCH provides multi-dimensional annotations that capture persuasion strategies from the scammer perspective, red flags from the user perspective, and severity levels reflecting potential harm and urgency. These annotations are produced through a human-AI collaborative pipeline that uses expert-labeled examples to calibrate and validate LLM-assisted annotation at scale. Our analysis shows that these signals vary significantly across scam types, highlighting substantial scam-type heterogeneity. We further use annotated evidence spans to generate cue-removed hard scam examples by rephrasing explicit persuasion and red-flag cues. Evaluating five LLM-based detectors on original and rewritten scams shows that all five detectors perform worse on cue-removed examples, and the performance drop persists on a human-validated subset, suggesting sensitivity to surface-level cues. A qualitative case study identifies five recurring failure patterns. SCAMBENCH provides a resource for scam detection, robustness evaluation, and scam education, publicly available at https://huggingface.co/datasets/Shouninger/ScamBench.
%U https://aclanthology.org/2026.findings-emnlp.404/
%P 8540-8563

Markdown (Informal)

[SCAMBENCH: A Multi-Dimensional Scam Benchmark through Human-AI Collaborative Data Construction](https://aclanthology.org/2026.findings-emnlp.404/) (Li et al., Findings 2026)

• SCAMBENCH: A Multi-Dimensional Scam Benchmark through Human-AI Collaborative Data Construction (Li et al., Findings 2026)

ACL

• Huaye Li, Siyu Huang, and Jinkyung Katie Park. 2026. SCAMBENCH: A Multi-Dimensional Scam Benchmark through Human-AI Collaborative Data Construction. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 8540–8563, Budapest, Hungary. Association for Computational Linguistics.