%0 Conference Proceedings %T Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models %A Ri, Ryokan %A Tsuruoka, Yoshimasa %Y Muresan, Smaranda %Y Nakov, Preslav %Y Villavicencio, Aline %S Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) %D 2022 %8 May %I Association for Computational Linguistics %C Dublin, Ireland %F ri-tsuruoka-2022-pretraining %R 10.18653/v1/2022.acl-long.504 %U https://aclanthology.org/2022.acl-long.504/ %U https://doi.org/10.18653/v1/2022.acl-long.504 %P 7302-7315