Minggui Duan
Author directory2026
Rule-Based Perturbation, Real Evaluation: How Far Do Fine-Tuned Encoders Go on Chinese Phonetically Cloaked Toxic Language?
Minggui Duan | Shijie Liu
Proceedings of the 10th Workshop on Online Abuse and Harms (WOAH)
Minggui Duan | Shijie Liu
Proceedings of the 10th Workshop on Online Abuse and Harms (WOAH)
Phonetic cloaking, the deliberate use of homophones, Pinyin, and numeric substitutions to disguise toxic content, poses a persistent challenge to Chinese toxic language detection. Existing work mainly relies on rule-based synthetic perturbations for benchmark evaluation, leaving uncertain how well such results transfer to naturally occurring cloaking. We present a controlled evaluation of fine-tuned encoders on PCR-ToxiCN, a dataset of naturally occurring, phonetically cloaked Chinese posts. Fine-tuning ChineseBERT and RoBERTa-wwm-ext on the CNTP synthetic perturbation dataset, we quantify synthetic-to-real performance gaps across four cloaking strategies. Synthetic perturbation training produces consistent but limited improvements, with diminishing returns as perturbation coverage expands. Hanzi (Chinese-character) replacement remains consistently difficult for both encoders, and few-shot fine-tuning on roughly 50 real examples provides only marginal improvement. The few-shot experiment, however, cannot distinguish representation limits from data scarcity or sampling effects. Overall, our findings caution against equating synthetic benchmark performance with real-world robustness.