Pradeep Murukannaiah

Author directory

2026

Hate speech lacks a single agreed definition across legal, social, and benchmark contexts, yet instruction-tuned large language models (LLMs) are increasingly used for hate speech detection. These systems are usually prompted with generic task instructions, so their predictions may reflect built-in moderation priors rather than a different operational definition from a given policy, benchmark, or jurisdiction. We investigate the ability of LLMs to align with specific hate speech definitions through zero-shot and few-shot prompting, by including the hate speech definition in the prompt. We opt for three definitions from different sources: dataset (HateCheck), social media (Reddit), and law (Bulgaria). We use HateCheck, a template-based hate speech diagnostic test set, to verify this alignment by relabeling the same test cases. We compare three LLMs, two definition inclusion conditions, and four prompting strategies. Our findings suggest that regardless of the prompting strategy, LLMs are not reliable in aligning properly with non-HateCheck definitions.