AutoML for NLP

Kevin Duh, Xuan Zhang


Abstract
Automated Machine Learning (AutoML) is an emerging field that has potential to impact how we build models in NLP. As an umbrella term that includes topics like hyperparameter optimization and neural architecture search, AutoML has recently become mainstream at major conferences such as NeurIPS, ICML, and ICLR. What does this mean to NLP? Currently, models are often built in an ad hoc process: we might borrow default hyperparameters from previous work and try a few variant architectures, but it is never guaranteed that final trained model is optimal. Automation can introduce rigor in this model-building process. This tutorial will summarize the main AutoML techniques and illustrate how to apply them to improve the NLP model-building process.
Anthology ID:
2023.eacl-tutorials.5
Volume:
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: Tutorial Abstracts
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Fabio Massimo Zanzotto, Sameer Pradhan
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
25–26
Language:
URL:
https://aclanthology.org/2023.eacl-tutorials.5
DOI:
10.18653/v1/2023.eacl-tutorials.5
Bibkey:
Cite (ACL):
Kevin Duh and Xuan Zhang. 2023. AutoML for NLP. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: Tutorial Abstracts, pages 25–26, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
AutoML for NLP (Duh & Zhang, EACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.eacl-tutorials.5.pdf
Video:
 https://aclanthology.org/2023.eacl-tutorials.5.mp4