TAAL: Target-Aware Active Learning

Kunal Kotian, Indranil Bhattacharya, Shikhar Gupta, Kaushik Pavani, Naval Bhandari, Sunny Dasgupta


Abstract
Pool-based active learning techniques have had success producing multi-class classifiers that achieve high accuracy with fewer labels com- pared to random labeling. However, in an industrial setting where we often have class-level business targets to achieve (e.g., 95% recall at 95% precision for each class), active learning techniques continue to acquire labels for classes that have already met their targets, thus consuming unnecessary manual annotations. We address this problem by proposing a framework called Target-Aware Active Learning that converts any active learning query strategy into its target-aware variant by leveraging the gap between each class’ current estimated accuracy and its corresponding business target. We show empirically that target-aware variants of state-of-the-art active learning techniques achieve business targets faster on 2 open-source image classification datasets and 2 proprietary product classification datasets.
Anthology ID:
2024.ecnlp-1.14
Volume:
Proceedings of the Seventh Workshop on e-Commerce and NLP @ LREC-COLING 2024
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Shervin Malmasi, Besnik Fetahu, Nicola Ueffing, Oleg Rokhlenko, Eugene Agichtein, Ido Guy
Venues:
ECNLP | WS
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
136–144
Language:
URL:
https://aclanthology.org/2024.ecnlp-1.14
DOI:
Bibkey:
Cite (ACL):
Kunal Kotian, Indranil Bhattacharya, Shikhar Gupta, Kaushik Pavani, Naval Bhandari, and Sunny Dasgupta. 2024. TAAL: Target-Aware Active Learning. In Proceedings of the Seventh Workshop on e-Commerce and NLP @ LREC-COLING 2024, pages 136–144, Torino, Italia. ELRA and ICCL.
Cite (Informal):
TAAL: Target-Aware Active Learning (Kotian et al., ECNLP-WS 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.ecnlp-1.14.pdf