On Orthogonality Constraints for Transformers Aston Zhang author Alvin Chan author Yi Tay author Jie Fu author Shuohang Wang author Shuai Zhang author Huajie Shao author Shuochao Yao author Roy Ka-Wei Lee author 2021-08 text Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) Chengqing Zong editor Fei Xia editor Wenjie Li editor Roberto Navigli editor Association for Computational Linguistics Online conference publication zhang-etal-2021-orthogonality 10.18653/v1/2021.acl-short.48 https://aclanthology.org/2021.acl-short.48/ 2021-08 375 382