Estimating Confidence of Predictions of Individual Classifiers and TheirEnsembles for the Genre Classification Task

Mikhail Lepekhin, Serge Sharoff


Abstract
Genre identification is a kind of non-topic text classification. The main difference between this task and topic classification is that genre, unlike topic, usually cannot be expressed just by some keywords and is defined as a functional space. Neural models based on pre-trained transformers, such as BERT or XLM-RoBERTa, demonstrate SOTA results in many NLP tasks, including non-topical classification. However, in many cases, their downstream application to very large corpora, such as those extracted from social media, can lead to unreliable results because of dataset shifts, when some raw texts do not match the profile of the training set. To mitigate this problem, we experiment with individual models as well as with their ensembles. To evaluate the robustness of all models we use a prediction confidence metric, which estimates the reliability of a prediction in the absence of a gold standard label. We can evaluate robustness via the confidence gap between the correctly classified texts and the misclassified ones on a labeled test corpus, higher gaps make it easier to identify whether a text is classified correctly. Our results show that for all of the classifiers tested in this study, there is a confidence gap, but for the ensembles, the gap is wider, meaning that ensembles are more robust than their individual models.
Anthology ID:
2022.lrec-1.642
Volume:
Proceedings of the Thirteenth Language Resources and Evaluation Conference
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
5974–5982
Language:
URL:
https://aclanthology.org/2022.lrec-1.642
DOI:
Bibkey:
Cite (ACL):
Mikhail Lepekhin and Serge Sharoff. 2022. Estimating Confidence of Predictions of Individual Classifiers and TheirEnsembles for the Genre Classification Task. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5974–5982, Marseille, France. European Language Resources Association.
Cite (Informal):
Estimating Confidence of Predictions of Individual Classifiers and TheirEnsembles for the Genre Classification Task (Lepekhin & Sharoff, LREC 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.lrec-1.642.pdf