<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="case-etal-2026-counterfactual">
    <titleInfo>
        <title>Counterfactual Robustness and the Calibration Trap in Antisemitism Detection</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Giacomo</namePart>
        <namePart type="family">Case</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yasmin</namePart>
        <namePart type="family">Chaqor</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Leonardo</namePart>
        <namePart type="family">Cartesegna</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 10th Workshop on Online Abuse and Harms (WOAH)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Agostina</namePart>
            <namePart type="family">Calabrese</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Thomas</namePart>
            <namePart type="family">Davidson</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Christine</namePart>
            <namePart type="family">de Kock</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Urja</namePart>
            <namePart type="family">Khurana</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marta</namePart>
            <namePart type="family">Marchiori Manerba</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Paloma</namePart>
            <namePart type="family">Piot</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zeerak</namePart>
            <namePart type="family">Talat</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-487-3</identifier>
    </relatedItem>
    <abstract>Classifiers for antisemitism detection inherit a shortcut from keyword-sampled training data: predictions flip when a Jewish identity term is rewritten to another religion, even though the gold label is preserved, conflating antisemitism with legitimate criticism of Israel. We audit this fragility jointly along three axes (positive-class performance, calibration, counterfactual stability) and report a previously unquantified tension. While four group-conditional post-hoc calibrators all improve global calibration (ECE), they inflate the soft counterfactual flip rate by 15–80%. A formal proposition shows intercept-free rescaling cannot change hard flips. Orbit-constrained calibration resolves it: it penalises calibrated probability differences on development-set identity-swap pairs, recovering the calibration gain (ECE \ensuremath-46%) with the hard rate unchanged; it is the only calibrator we test that lowers ECE without inflating soft flips. Separately, we attack invariance at training time: CCI (Counterfactual Consistency Invariance) adds a Jensen–Shannon consistency penalty between each tweet and its religiously rewritten counterfactuals. This cuts the hard flip rate by 49–73% across three encoder backbones and two model families, while preserving or improving F1+ (up to 0.72). Plain counterfactual augmentation reaches a lower flip rate at 3× the training-set size; raising CCI’s multiplier closes that gap at a matched predicted-positive rate. Three prompted 7–8B LLMs cap at F1+ 0.45 and flip an order of magnitude more often. Finally, Integrated Gradients show identity-token attribution nearly unchanged: the gain comes from coupling pair scores, not from suppressing identity salience.</abstract>
    <identifier type="citekey">case-etal-2026-counterfactual</identifier>
    <location>
        <url>https://aclanthology.org/2026.woah-1.24/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>273</start>
            <end>283</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Counterfactual Robustness and the Calibration Trap in Antisemitism Detection
%A Case, Giacomo
%A Chaqor, Yasmin
%A Cartesegna, Leonardo
%Y Calabrese, Agostina
%Y Davidson, Thomas
%Y de Kock, Christine
%Y Khurana, Urja
%Y Marchiori Manerba, Marta
%Y Piot, Paloma
%Y Talat, Zeerak
%S Proceedings of the 10th Workshop on Online Abuse and Harms (WOAH)
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-487-3
%F case-etal-2026-counterfactual
%X Classifiers for antisemitism detection inherit a shortcut from keyword-sampled training data: predictions flip when a Jewish identity term is rewritten to another religion, even though the gold label is preserved, conflating antisemitism with legitimate criticism of Israel. We audit this fragility jointly along three axes (positive-class performance, calibration, counterfactual stability) and report a previously unquantified tension. While four group-conditional post-hoc calibrators all improve global calibration (ECE), they inflate the soft counterfactual flip rate by 15–80%. A formal proposition shows intercept-free rescaling cannot change hard flips. Orbit-constrained calibration resolves it: it penalises calibrated probability differences on development-set identity-swap pairs, recovering the calibration gain (ECE \ensuremath-46%) with the hard rate unchanged; it is the only calibrator we test that lowers ECE without inflating soft flips. Separately, we attack invariance at training time: CCI (Counterfactual Consistency Invariance) adds a Jensen–Shannon consistency penalty between each tweet and its religiously rewritten counterfactuals. This cuts the hard flip rate by 49–73% across three encoder backbones and two model families, while preserving or improving F1+ (up to 0.72). Plain counterfactual augmentation reaches a lower flip rate at 3× the training-set size; raising CCI’s multiplier closes that gap at a matched predicted-positive rate. Three prompted 7–8B LLMs cap at F1+ 0.45 and flip an order of magnitude more often. Finally, Integrated Gradients show identity-token attribution nearly unchanged: the gain comes from coupling pair scores, not from suppressing identity salience.
%U https://aclanthology.org/2026.woah-1.24/
%P 273-283

Markdown (Informal)

[Counterfactual Robustness and the Calibration Trap in Antisemitism Detection](https://aclanthology.org/2026.woah-1.24/) (Case et al., WOAH 2026)

• Counterfactual Robustness and the Calibration Trap in Antisemitism Detection (Case et al., WOAH 2026)

ACL

• Giacomo Case, Yasmin Chaqor, and Leonardo Cartesegna. 2026. Counterfactual Robustness and the Calibration Trap in Antisemitism Detection. In Proceedings of the 10th Workshop on Online Abuse and Harms (WOAH), pages 273–283, Budapest, Hungary. Association for Computational Linguistics.