Initializing neural networks for hierarchical multi-label text classification

Simon Baker, Anna Korhonen


Abstract
Many tasks in the biomedical domain require the assignment of one or more predefined labels to input text, where the labels are a part of a hierarchical structure (such as a taxonomy). The conventional approach is to use a one-vs.-rest (OVR) classification setup, where a binary classifier is trained for each label in the taxonomy or ontology where all instances not belonging to the class are considered negative examples. The main drawbacks to this approach are that dependencies between classes are not leveraged in the training and classification process, and the additional computational cost of training parallel classifiers. In this paper, we apply a new method for hierarchical multi-label text classification that initializes a neural network model final hidden layer such that it leverages label co-occurrence relations such as hypernymy. This approach elegantly lends itself to hierarchical classification. We evaluated this approach using two hierarchical multi-label text classification tasks in the biomedical domain using both sentence- and document-level classification. Our evaluation shows promising results for this approach.
Anthology ID:
W17-2339
Volume:
BioNLP 2017
Month:
August
Year:
2017
Address:
Vancouver, Canada,
Editors:
Kevin Bretonnel Cohen, Dina Demner-Fushman, Sophia Ananiadou, Junichi Tsujii
Venue:
BioNLP
SIG:
SIGBIOMED
Publisher:
Association for Computational Linguistics
Note:
Pages:
307–315
Language:
URL:
https://aclanthology.org/W17-2339
DOI:
10.18653/v1/W17-2339
Bibkey:
Cite (ACL):
Simon Baker and Anna Korhonen. 2017. Initializing neural networks for hierarchical multi-label text classification. In BioNLP 2017, pages 307–315, Vancouver, Canada,. Association for Computational Linguistics.
Cite (Informal):
Initializing neural networks for hierarchical multi-label text classification (Baker & Korhonen, BioNLP 2017)
Copy Citation:
PDF:
https://aclanthology.org/W17-2339.pdf