Batin Örene
Author directory2026
Context-Aware Evidence Minimization for Privacy-Conscious Harmful-Speech Dataset Release
Batin Örene
Proceedings of the 10th Workshop on Online Abuse and Harms (WOAH)
Batin Örene
Proceedings of the 10th Workshop on Online Abuse and Harms (WOAH)
Harmful-speech datasets drive moderation research but are risky to release as raw text. We ask how much source-text exposure curators can remove while preserving evidence for model comparison and error analysis. Releasing only classifier-important tokens corrupts the data: an independent, lexicon-free LLM judge recovers the targeted group from such excerpts only 36% of the time, and reads 53% of non-hateful excerpts as hateful. We introduce I-CEM, a deterministic evidence-minimization baseline that masks recorded synthetic identifiers, retains tokens whose removal lowers a frozen classifier’s harmful-speech score, and expands retained spans only when rules detect target, harm, negation, quotation, or counterspeech context. We evaluate on HateXplain, HateCheck, and Civil Comments with controlled synthetic PII injection. Our primary result is target recovery: I-CEM raises it from 58% to 67% over a token-matched fixed window (p.001), and the gain replicates against the dataset’s gold target labels (50% to 56%, p.001); I-CEM also more than halves the importance-only misreading (25%). Secondary gains are directional: hate-label fidelity improves modestly (90% vs. 88%, p=.04) but does not hold against gold labels, and the stance advantage over a matched window is not significant. On 5,000 HateXplain examples I-CEM retains about half of source tokens and stays within 0.02 positive-class F1 of a fixed importance window, but releases more text than that window when its context rules fire, a trade curators opt into. Masking every recorded injected identifier validates the masking pipeline; it is not evidence of real-world anonymization. I-CEM is a deterministic context-recovery baseline that measurably reduces label corruption from evidence-only release.