Cloze-driven Pretraining of Self-attention Networks

Cloze-driven Pretraining of Self-attention Networks Alexei Baevski author Sergey Edunov author Yinhan Liu author Luke Zettlemoyer author Michael Auli author 2019-11 text Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) Kentaro Inui editor Jing Jiang editor Vincent Ng editor Xiaojun Wan editor Association for Computational Linguistics Hong Kong, China conference publication baevski-etal-2019-cloze 10.18653/v1/D19-1539 https://aclanthology.org/D19-1539/ 2019-11 5360 5369