Parham Bateni
Author directory2026
Tailoring In-Context Learning Techniques for Definition-Based Hate Speech Detection in Large Language Models
Parham Bateni | Pradeep Murukannaiah | Urja Khurana
Proceedings of the 10th Workshop on Online Abuse and Harms (WOAH)
Parham Bateni | Pradeep Murukannaiah | Urja Khurana
Proceedings of the 10th Workshop on Online Abuse and Harms (WOAH)
Hate speech lacks a single agreed definition across legal, social, and benchmark contexts, yet instruction-tuned large language models (LLMs) are increasingly used for hate speech detection. These systems are usually prompted with generic task instructions, so their predictions may reflect built-in moderation priors rather than a different operational definition from a given policy, benchmark, or jurisdiction. We investigate the ability of LLMs to align with specific hate speech definitions through zero-shot and few-shot prompting, by including the hate speech definition in the prompt. We opt for three definitions from different sources: dataset (HateCheck), social media (Reddit), and law (Bulgaria). We use HateCheck, a template-based hate speech diagnostic test set, to verify this alignment by relabeling the same test cases. We compare three LLMs, two definition inclusion conditions, and four prompting strategies. Our findings suggest that regardless of the prompting strategy, LLMs are not reliable in aligning properly with non-HateCheck definitions.