Automatic Discovery of Heterogeneous Machine Learning Pipelines: An Application to Natural Language Processing

Suilan Estevez-Velarde, Yoan Gutiérrez, Andres Montoyo, Yudivián Almeida Cruz


Abstract
This paper presents AutoGOAL, a system for automatic machine learning (AutoML) that uses heterogeneous techniques. In contrast with existing AutoML approaches, our contribution can automatically build machine learning pipelines that combine techniques and algorithms from different frameworks, including shallow classifiers, natural language processing tools, and neural networks. We define the heterogeneous AutoML optimization problem as the search for the best sequence of algorithms that transforms specific input data into the desired output. This provides a novel theoretical and practical approach to AutoML. Our proposal is experimentally evaluated in diverse machine learning problems and compared with alternative approaches, showing that it is competitive with other AutoML alternatives in standard benchmarks. Furthermore, it can be applied to novel scenarios, such as several NLP tasks, where existing alternatives cannot be directly deployed. The system is freely available and includes in-built compatibility with a large number of popular machine learning frameworks, which makes our approach useful for solving practical problems with relative ease and effort.
Anthology ID:
2020.coling-main.317
Volume:
Proceedings of the 28th International Conference on Computational Linguistics
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Donia Scott, Nuria Bel, Chengqing Zong
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
3558–3568
Language:
URL:
https://aclanthology.org/2020.coling-main.317
DOI:
10.18653/v1/2020.coling-main.317
Bibkey:
Cite (ACL):
Suilan Estevez-Velarde, Yoan Gutiérrez, Andres Montoyo, and Yudivián Almeida Cruz. 2020. Automatic Discovery of Heterogeneous Machine Learning Pipelines: An Application to Natural Language Processing. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3558–3568, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Cite (Informal):
Automatic Discovery of Heterogeneous Machine Learning Pipelines: An Application to Natural Language Processing (Estevez-Velarde et al., COLING 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.coling-main.317.pdf