Mask More and Mask Later: Efficient Pre-training of Masked Language Models by Disentangling the [MASK] Token

Mask More and Mask Later: Efficient Pre-training of Masked Language Models by Disentangling the [MASK] Token Baohao Liao author David Thulke author Sanjika Hewavitharana author Hermann Ney author Christof Monz author 2022-12 text Findings of the Association for Computational Linguistics: EMNLP 2022 Yoav Goldberg editor Zornitsa Kozareva editor Yue Zhang editor Association for Computational Linguistics Abu Dhabi, United Arab Emirates conference publication liao-etal-2022-mask 10.18653/v1/2022.findings-emnlp.106 https://aclanthology.org/2022.findings-emnlp.106/ 2022-12 1478 1492