AustroTox: A Dataset for Target-Based Austrian German Offensive Language Detection

Pia Pachinger, Janis Goldzycher, Anna Planitzer, Wojciech Kusa, Allan Hanbury, Julia Neidhardt


Abstract
Model interpretability in toxicity detection greatly profits from token-level annotations. However, currently, such annotations are only available in English. We introduce a dataset annotated for offensive language detection sourced from a news forum, notable for its incorporation of the Austrian German dialect, comprising 4,562 user comments. In addition to binary offensiveness classification, we identify spans within each comment constituting vulgar language or representing targets of offensive statements. We evaluate fine-tuned Transformer models as well as large language models in a zero- and few-shot fashion. The results indicate that while fine-tuned models excel in detecting linguistic peculiarities such as vulgar dialect, large language models demonstrate superior performance in detecting offensiveness in AustroTox.
Anthology ID:
2024.findings-acl.713
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
11990–12001
Language:
URL:
https://aclanthology.org/2024.findings-acl.713
DOI:
Bibkey:
Cite (ACL):
Pia Pachinger, Janis Goldzycher, Anna Planitzer, Wojciech Kusa, Allan Hanbury, and Julia Neidhardt. 2024. AustroTox: A Dataset for Target-Based Austrian German Offensive Language Detection. In Findings of the Association for Computational Linguistics ACL 2024, pages 11990–12001, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
AustroTox: A Dataset for Target-Based Austrian German Offensive Language Detection (Pachinger et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-acl.713.pdf