From Consensus to Split Decisions: ABC-Stratified Sentiment in Holocaust Oral Histories

Daban Q. Jaff


Abstract
Polarity detection becomes substantially more challenging under domain shift, particularly in heterogeneous long-form narratives with complex discourse structure, such as Holocaust oral histories. This paper presents a corpus-scale diagnostic study of off-the-shelf sentiment classifiers on Holocaust oral histories, using three pretrained transformer-based polarity classifiers over a corpus comprising 107,304 utterances and 579,013 sentences. After assembling model outputs, we introduce an agreement-based stability taxonomy (ABC) to stratify inter-model output stability. We report pairwise percent agreement, Cohen’s κ, Fleiss’ κ, and row-normalized confusion matrices to localize systematic disagreement. As an external convergent descriptive signal, we apply a T5-based emotion classifier to stratified samples from each agreement stratum to compare emotion distributions across strata. The combination of multi-model label triangulation and the ABC taxonomy provides a cautious, interpretable framework for characterizing where and how sentiment models diverge in sensitive historical narratives. Inter-model agreement is low to moderate overall and is driven primarily by boundary decisions around neutrality.
Anthology ID:
2026.htres-2.4
Volume:
Proceedings of The Second Workshop on Holocaust Testimonies as Language Resources (HTRes)
Month:
May
Year:
2026
Address:
Palma, Mallorca (Spain)
Editors:
Isuri Anuradha, Martin Wynne
Venues:
htres | WS
SIG:
Publisher:
ELRA Language Resources Association (ELRA)
Note:
Pages:
29–36
Language:
External URL:
https://lrec.elra.info/lrec2026-ws-htres-04
DOI:
10.63317/2t969u9fy4gf
Bibkey:
Cite (ACL):
Daban Q. Jaff. 2026. From Consensus to Split Decisions: ABC-Stratified Sentiment in Holocaust Oral Histories. In Proceedings of The Second Workshop on Holocaust Testimonies as Language Resources (HTRes), pages 29–36, Palma, Mallorca (Spain). ELRA Language Resources Association (ELRA).
Cite (Informal):
From Consensus to Split Decisions: ABC-Stratified Sentiment in Holocaust Oral Histories (Jaff, htres 2026)
Copy Citation: