Learned Transformer Position Embeddings Have a Low-Dimensional Structure

Learned Transformer Position Embeddings Have a Low-Dimensional Structure Ulme Wennberg author Gustav Henter author 2024-08 text Proceedings of the 9th Workshop on Representation Learning for NLP (RepL4NLP-2024) Chen Zhao editor Marius Mosbach editor Pepa Atanasova editor Seraphina Goldfarb-Tarrent editor Peter Hase editor Arian Hosseini editor Maha Elbayad editor Sandro Pezzelle editor Maximilian Mozes editor Association for Computational Linguistics Bangkok, Thailand conference publication wennberg-henter-2024-learned https://aclanthology.org/2024.repl4nlp-1.17/ 2024-08 237 244