Randomized Positional Encodings Boost Length Generalization of Transformers

Randomized Positional Encodings Boost Length Generalization of Transformers Anian Ruoss author Grégoire Delétang author Tim Genewein author Jordi Grau-Moya author Róbert Csordás author Mehdi Bennani author Shane Legg author Joel Veness author 2023-07 text Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) Anna Rogers editor Jordan Boyd-Graber editor Naoaki Okazaki editor Association for Computational Linguistics Toronto, Canada conference publication ruoss-etal-2023-randomized 10.18653/v1/2023.acl-short.161 https://aclanthology.org/2023.acl-short.161/ 2023-07 1889 1903