Absolute Position Embedding Learns Sinusoid-like Waves for Attention Based on Relative Position

Absolute Position Embedding Learns Sinusoid-like Waves for Attention Based on Relative Position Yuji Yamamoto author Takuya Matsuzaki author 2023-12 text Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing Houda Bouamor editor Juan Pino editor Kalika Bali editor Association for Computational Linguistics Singapore conference publication yamamoto-matsuzaki-2023-absolute 10.18653/v1/2023.emnlp-main.2 https://aclanthology.org/2023.emnlp-main.2/ 2023-12 15 28