SemEval-2021 Task 5: Toxic Spans Detection

John Pavlopoulos, Jeffrey Sorensen, Léo Laugier, Ion Androutsopoulos


Abstract
The Toxic Spans Detection task of SemEval-2021 required participants to predict the spans of toxic posts that were responsible for the toxic label of the posts. The task could be addressed as supervised sequence labeling, using training data with gold toxic spans provided by the organisers. It could also be treated as rationale extraction, using classifiers trained on potentially larger external datasets of posts manually annotated as toxic or not, without toxic span annotations. For the supervised sequence labeling approach and evaluation purposes, posts previously labeled as toxic were crowd-annotated for toxic spans. Participants submitted their predicted spans for a held-out test set and were scored using character-based F1. This overview summarises the work of the 36 teams that provided system descriptions.
Anthology ID:
2021.semeval-1.6
Volume:
Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)
Month:
August
Year:
2021
Address:
Online
Editors:
Alexis Palmer, Nathan Schneider, Natalie Schluter, Guy Emerson, Aurelie Herbelot, Xiaodan Zhu
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
59–69
Language:
URL:
https://aclanthology.org/2021.semeval-1.6
DOI:
10.18653/v1/2021.semeval-1.6
Bibkey:
Cite (ACL):
John Pavlopoulos, Jeffrey Sorensen, Léo Laugier, and Ion Androutsopoulos. 2021. SemEval-2021 Task 5: Toxic Spans Detection. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 59–69, Online. Association for Computational Linguistics.
Cite (Informal):
SemEval-2021 Task 5: Toxic Spans Detection (Pavlopoulos et al., SemEval 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.semeval-1.6.pdf
Video:
 https://aclanthology.org/2021.semeval-1.6.mp4
Data
Civil Comments