Sebastiano Vecellio Salto
2024
Delving into Qualitative Implications of Synthetic Data for Hate Speech Detection
Camilla Casula
|
Sebastiano Vecellio Salto
|
Alan Ramponi
|
Sara Tonelli
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
The use of synthetic data for training models for a variety of NLP tasks is now widespread. However, previous work reports mixed results with regards to its effectiveness on highly subjective tasks such as hate speech detection. In this paper, we present an in-depth qualitative analysis of the potential and specific pitfalls of synthetic data for hate speech detection in English, with 3,500 manually annotated examples. We show that, across different models, synthetic data created through paraphrasing gold texts can improve out-of-distribution robustness from a computational standpoint. However, this comes at a cost: synthetic data fails to reliably reflect the characteristics of real-world data on a number of linguistic dimensions, it results in drastically different class distributions, and it heavily reduces the representation of both specific identity groups and intersectional hate.
Search