Orthogonal Matching Pursuit for Text Classification

Konstantinos Skianis, Nikolaos Tziortziotis, Michalis Vazirgiannis


Abstract
In text classification, the problem of overfitting arises due to the high dimensionality, making regularization essential. Although classic regularizers provide sparsity, they fail to return highly accurate models. On the contrary, state-of-the-art group-lasso regularizers provide better results at the expense of low sparsity. In this paper, we apply a greedy variable selection algorithm, called Orthogonal Matching Pursuit, for the text classification task. We also extend standard group OMP by introducing overlapping Group OMP to handle overlapping groups of features. Empirical analysis verifies that both OMP and overlapping GOMP constitute powerful regularizers, able to produce effective and very sparse models. Code and data are available online.
Anthology ID:
W18-6113
Volume:
Proceedings of the 2018 EMNLP Workshop W-NUT: The 4th Workshop on Noisy User-generated Text
Month:
November
Year:
2018
Address:
Brussels, Belgium
Editors:
Wei Xu, Alan Ritter, Tim Baldwin, Afshin Rahimi
Venue:
WNUT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
93–103
Language:
URL:
https://aclanthology.org/W18-6113
DOI:
10.18653/v1/W18-6113
Bibkey:
Cite (ACL):
Konstantinos Skianis, Nikolaos Tziortziotis, and Michalis Vazirgiannis. 2018. Orthogonal Matching Pursuit for Text Classification. In Proceedings of the 2018 EMNLP Workshop W-NUT: The 4th Workshop on Noisy User-generated Text, pages 93–103, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Orthogonal Matching Pursuit for Text Classification (Skianis et al., WNUT 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-6113.pdf
Code
 y3nk0/OMP-for-Text-Classification