Attend First, Consolidate Later: On the Importance of Attention in Different LLM Layers

Attend First, Consolidate Later: On the Importance of Attention in Different LLM Layers Amit Ben Artzy author Roy Schwartz author 2024-11 text Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP Yonatan Belinkov editor Najoung Kim editor Jaap Jumelet editor Hosein Mohebbi editor Aaron Mueller editor Hanjie Chen editor Association for Computational Linguistics Miami, Florida, US conference publication artzy-schwartz-2024-attend 10.18653/v1/2024.blackboxnlp-1.10 https://aclanthology.org/2024.blackboxnlp-1.10/ 2024-11 177 184