ToxiCraft: A Novel Framework for Synthetic Generation of Harmful Information

Zheng Hui, Zhaoxiao Guo, Hang Zhao, Juanyong Duan, Congrui Huang


Abstract
In different NLP tasks, detecting harmful content is crucial for online environments, especially with the growing influence of social media. However, previous research has two main issues: 1) a lack of data in low-resource settings, and 2) inconsistent definitions and criteria for judging harmful content, requiring classification models to be robust to spurious features and diverse. We propose Toxicraft, a novel framework for synthesizing datasets of harmful information to address these weaknesses. With only a small amount of seed data, our framework can generate a wide variety of synthetic, yet remarkably realistic, examples of toxic information. Experimentation across various datasets showcases a notable enhancement in detection model robustness and adaptability, surpassing or close to the gold labels.
Anthology ID:
2024.findings-emnlp.970
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2024
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
16632–16647
Language:
URL:
https://aclanthology.org/2024.findings-emnlp.970
DOI:
Bibkey:
Cite (ACL):
Zheng Hui, Zhaoxiao Guo, Hang Zhao, Juanyong Duan, and Congrui Huang. 2024. ToxiCraft: A Novel Framework for Synthetic Generation of Harmful Information. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 16632–16647, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
ToxiCraft: A Novel Framework for Synthetic Generation of Harmful Information (Hui et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-emnlp.970.pdf