Modeling the Severity of Complaints in Social Media

Mali Jin, Nikolaos Aletras


Abstract
The speech act of complaining is used by humans to communicate a negative mismatch between reality and expectations as a reaction to an unfavorable situation. Linguistic theory of pragmatics categorizes complaints into various severity levels based on the face-threat that the complainer is willing to undertake. This is particularly useful for understanding the intent of complainers and how humans develop suitable apology strategies. In this paper, we study the severity level of complaints for the first time in computational linguistics. To facilitate this, we enrich a publicly available data set of complaints with four severity categories and train different transformer-based networks combined with linguistic information achieving 55.7 macro F1. We also jointly model binary complaint classification and complaint severity in a multi-task setting achieving new state-of-the-art results on binary complaint detection reaching up to 88.2 macro F1. Finally, we present a qualitative analysis of the behavior of our models in predicting complaint severity levels.
Anthology ID:
2021.naacl-main.180
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
June
Year:
2021
Address:
Online
Editors:
Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, Yichao Zhou
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2264–2274
Language:
URL:
https://aclanthology.org/2021.naacl-main.180
DOI:
10.18653/v1/2021.naacl-main.180
Bibkey:
Cite (ACL):
Mali Jin and Nikolaos Aletras. 2021. Modeling the Severity of Complaints in Social Media. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2264–2274, Online. Association for Computational Linguistics.
Cite (Informal):
Modeling the Severity of Complaints in Social Media (Jin & Aletras, NAACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.naacl-main.180.pdf
Video:
 https://aclanthology.org/2021.naacl-main.180.mp4
Code
 mali726/Complaint-Severity