BERT-Flow-VAE: A Weakly-supervised Model for Multi-Label Text Classification

Ziwen Liu, Josep Grau-Bove, Scott Allan Orr


Abstract
Multi-label Text Classification (MLTC) is the task of categorizing documents into one or more topics. Considering the large volumes of data and varying domains of such tasks, fully supervised learning requires manually fully annotated datasets which is costly and time-consuming. In this paper, we propose BERT-Flow-VAE (BFV), a Weakly-Supervised Multi-Label Text Classification (WSMLTC) model that reduces the need for full supervision. This new model (1) produces BERT sentence embeddings and calibrates them using a flow model, (2) generates an initial topic-document matrix by averaging results of a seeded sparse topic model and a textual entailment model which only require surface name of topics and 4-6 seed words per topic, and (3) adopts a VAE framework to reconstruct the embeddings under the guidance of the topic-document matrix. Finally, (4) it uses the means produced by the encoder model in the VAE architecture as predictions for MLTC. Experimental results on 6 multi-label datasets show that BFV can substantially outperform other baseline WSMLTC models in key metrics and achieve approximately 84% performance of a fully-supervised model.
Anthology ID:
2022.coling-1.104
Volume:
Proceedings of the 29th International Conference on Computational Linguistics
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Editors:
Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, Seung-Hoon Na
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
1203–1220
Language:
URL:
https://aclanthology.org/2022.coling-1.104
DOI:
Bibkey:
Cite (ACL):
Ziwen Liu, Josep Grau-Bove, and Scott Allan Orr. 2022. BERT-Flow-VAE: A Weakly-supervised Model for Multi-Label Text Classification. In Proceedings of the 29th International Conference on Computational Linguistics, pages 1203–1220, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Cite (Informal):
BERT-Flow-VAE: A Weakly-supervised Model for Multi-Label Text Classification (Liu et al., COLING 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.coling-1.104.pdf