On the Transformer Growth for Progressive BERT Training

On the Transformer Growth for Progressive BERT Training Xiaotao Gu author Liyuan Liu author Hongkun Yu author Jing Li author Chen Chen author Jiawei Han author 2021-06 text Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Kristina Toutanova editor Anna Rumshisky editor Luke Zettlemoyer editor Dilek Hakkani-Tur editor Iz Beltagy editor Steven Bethard editor Ryan Cotterell editor Tanmoy Chakraborty editor Yichao Zhou editor Association for Computational Linguistics Online conference publication gu-etal-2021-transformer 10.18653/v1/2021.naacl-main.406 https://aclanthology.org/2021.naacl-main.406/ 2021-06 5174 5180