Zero- and Few-Shot NLP with Pretrained Language Models

Iz Beltagy, Arman Cohan, Robert Logan IV, Sewon Min, Sameer Singh


Abstract
The ability to efficiently learn from little-to-no data is critical to applying NLP to tasks where data collection is costly or otherwise difficult. This is a challenging setting both academically and practically—particularly because training neutral models typically require large amount of labeled data. More recently, advances in pretraining on unlabelled data have brought up the potential of better zero-shot or few-shot learning (Devlin et al., 2019; Brown et al., 2020). In particular, over the past year, a great deal of research has been conducted to better learn from limited data using large-scale language models. In this tutorial, we aim at bringing interested NLP researchers up to speed about the recent and ongoing techniques for zero- and few-shot learning with pretrained language models. Additionally, our goal is to reveal new research opportunities to the audience, which will hopefully bring us closer to address existing challenges in this domain.
Anthology ID:
2022.acl-tutorials.6
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Luciana Benotti, Naoaki Okazaki, Yves Scherrer, Marcos Zampieri
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
32–37
Language:
URL:
https://aclanthology.org/2022.acl-tutorials.6
DOI:
10.18653/v1/2022.acl-tutorials.6
Bibkey:
Cite (ACL):
Iz Beltagy, Arman Cohan, Robert Logan IV, Sewon Min, and Sameer Singh. 2022. Zero- and Few-Shot NLP with Pretrained Language Models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, pages 32–37, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Zero- and Few-Shot NLP with Pretrained Language Models (Beltagy et al., ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-tutorials.6.pdf