ConvAI at SemEval-2019 Task 6: Offensive Language Identification and Categorization with Perspective and BERT

John Pavlopoulos, Nithum Thain, Lucas Dixon, Ion Androutsopoulos


Abstract
This paper presents the application of two strong baseline systems for toxicity detection and evaluates their performance in identifying and categorizing offensive language in social media. PERSPECTIVE is an API, that serves multiple machine learning models for the improvement of conversations online, as well as a toxicity detection system, trained on a wide variety of comments from platforms across the Internet. BERT is a recently popular language representation model, fine tuned per task and achieving state of the art performance in multiple NLP tasks. PERSPECTIVE performed better than BERT in detecting toxicity, but BERT was much better in categorizing the offensive type. Both baselines were ranked surprisingly high in the SEMEVAL-2019 OFFENSEVAL competition, PERSPECTIVE in detecting an offensive post (12th) and BERT in categorizing it (11th). The main contribution of this paper is the assessment of two strong baselines for the identification (PERSPECTIVE) and the categorization (BERT) of offensive language with little or no additional training data.
Anthology ID:
S19-2102
Volume:
Proceedings of the 13th International Workshop on Semantic Evaluation
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota, USA
Editors:
Jonathan May, Ekaterina Shutova, Aurelie Herbelot, Xiaodan Zhu, Marianna Apidianaki, Saif M. Mohammad
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
571–576
Language:
URL:
https://aclanthology.org/S19-2102
DOI:
10.18653/v1/S19-2102
Bibkey:
Cite (ACL):
John Pavlopoulos, Nithum Thain, Lucas Dixon, and Ion Androutsopoulos. 2019. ConvAI at SemEval-2019 Task 6: Offensive Language Identification and Categorization with Perspective and BERT. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 571–576, Minneapolis, Minnesota, USA. Association for Computational Linguistics.
Cite (Informal):
ConvAI at SemEval-2019 Task 6: Offensive Language Identification and Categorization with Perspective and BERT (Pavlopoulos et al., SemEval 2019)
Copy Citation:
PDF:
https://aclanthology.org/S19-2102.pdf