Transformer Language Models without Positional Encodings Still Learn Positional Information

Transformer Language Models without Positional Encodings Still Learn Positional Information Adi Haviv author Ori Ram author Ofir Press author Peter Izsak author Omer Levy author 2022-12 text Findings of the Association for Computational Linguistics: EMNLP 2022 Yoav Goldberg editor Zornitsa Kozareva editor Yue Zhang editor Association for Computational Linguistics Abu Dhabi, United Arab Emirates conference publication haviv-etal-2022-transformer 10.18653/v1/2022.findings-emnlp.99 https://aclanthology.org/2022.findings-emnlp.99/ 2022-12 1382 1390