Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT

Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT James Lee-Thorp author Joshua Ainslie author 2022-12 text Findings of the Association for Computational Linguistics: EMNLP 2022 Yoav Goldberg editor Zornitsa Kozareva editor Yue Zhang editor Association for Computational Linguistics Abu Dhabi, United Arab Emirates conference publication lee-thorp-ainslie-2022-sparse 10.18653/v1/2022.findings-emnlp.5 https://aclanthology.org/2022.findings-emnlp.5/ 2022-12 58 75