Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification

Varsha Suresh, Desmond Ong


Abstract
Fine-grained classification involves dealing with datasets with larger number of classes with subtle differences between them. Guiding the model to focus on differentiating dimensions between these commonly confusable classes is key to improving performance on fine-grained tasks. In this work, we analyse the contrastive fine-tuning of pre-trained language models on two fine-grained text classification tasks, emotion classification and sentiment analysis. We adaptively embed class relationships into a contrastive objective function to help differently weigh the positives and negatives, and in particular, weighting closely confusable negatives more than less similar negative examples. We find that Label-aware Contrastive Loss outperforms previous contrastive methods, in the presence of larger number and/or more confusable classes, and helps models to produce output distributions that are more differentiated.
Anthology ID:
2021.emnlp-main.359
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4381–4394
Language:
URL:
https://aclanthology.org/2021.emnlp-main.359
DOI:
10.18653/v1/2021.emnlp-main.359
Bibkey:
Cite (ACL):
Varsha Suresh and Desmond Ong. 2021. Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4381–4394, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification (Suresh & Ong, EMNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-main.359.pdf
Video:
 https://aclanthology.org/2021.emnlp-main.359.mp4
Code
 varsha33/lcl_loss
Data
GoEmotionsISEARSSTSST-2SST-5