KOCOH: Korean Context-Dependent Hate Speech Dataset

Eunah Park, Sanghoun Song


Abstract
We introduce the KOrean COntext-dependent Hate speech dataset (KOCOH) to evaluate large language models’ ability to detect context-dependent hate speech in Korean. KOCOH consists of 3,000 context-comment pairs collected from Korean online communities (Dcinside, FMkorea) with detailed annotations, including labels for hate speech and hate target groups. We assess the context-dependent hate speech detection capabilities of both humans and 11 state-of-the-art large language models, including GPT-5, Claude Sonnet 4, and Gemini 2.5 Flash. Our results show that humans outperform language models, with GPT-5 achieving the highest performance among the evaluated models. While humans demonstrate balanced recall and specificity, language models generally show significantly higher specificity compared to recall. The performance of both humans and models is affected by factors such as Honam-related vocabulary and sentiment polarity. This study contributes resources to Korean hate speech research and empirically demonstrates the performance gap between humans and language models. Through both quantitative and qualitative analyses, we explore the similarities and differences between humans and language models, offering insights for future developments in language models and AI ethics research. KOCOH is available at https://github.com/eparkatgithub/KOCOH.
Anthology ID:
2026.lrec-1.324
Volume:
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Month:
May
Year:
2026
Address:
Palma de Mallorca, Spain
Editors:
Stelios Piperidis, Núria Bel, Henk van den Heuvel, Nancy Ide, Simon Krek, Antonio Toral
Venue:
LREC
SIG:
Publisher:
ELRA Language Resource Association
Note:
Pages:
4103–4114
Language:
External URL:
https://lrec.elra.info/lrec2026-main-324
DOI:
10.63317/5pnu2jn6awun
Bibkey:
Cite (ACL):
Eunah Park and Sanghoun Song. 2026. KOCOH: Korean Context-Dependent Hate Speech Dataset. In Proceedings of the Fifteenth Language Resources and Evaluation Conference, pages 4103–4114, Palma de Mallorca, Spain. ELRA Language Resource Association.
Cite (Informal):
KOCOH: Korean Context-Dependent Hate Speech Dataset (Park & Song, LREC 2026)
Copy Citation: