Scalable Efficient Training of Large Language Models with Low-dimensional Projected Attention

Scalable Efficient Training of Large Language Models with Low-dimensional Projected Attention Xingtai Lv author Ning Ding author Kaiyan Zhang author Ermo Hua author Ganqu Cui author Bowen Zhou author 2024-11 text Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing Yaser Al-Onaizan editor Mohit Bansal editor Yun-Nung Chen editor Association for Computational Linguistics Miami, Florida, USA conference publication lv-etal-2024-scalable 10.18653/v1/2024.emnlp-main.808 https://aclanthology.org/2024.emnlp-main.808/ 2024-11 14588 14599