%0 Conference Proceedings %T Investigating the Quality of Static Anchor Embeddings from Transformers for Under-Resourced Languages %A Singh, Pranaydeep %A De Clercq, Orphee %A Lefever, Els %Y Melero, Maite %Y Sakti, Sakriani %Y Soria, Claudia %S Proceedings of the 1st Annual Meeting of the ELRA/ISCA Special Interest Group on Under-Resourced Languages %D 2022 %8 June %I European Language Resources Association %C Marseille, France %F singh-etal-2022-investigating %X This paper reports on experiments for cross-lingual transfer using the anchor-based approach of Schuster et al. (2019) for English and a low-resourced language, namely Hindi. For the sake of comparison, we also evaluate the approach on three very different higher-resourced languages, viz. Dutch, Russian and Chinese. Initially designed for ELMo embeddings, we analyze the approach for the more recent BERT family of transformers for a variety of tasks, both mono and cross-lingual. The results largely prove that like most other cross-lingual transfer approaches, the static anchor approach is underwhelming for the low-resource language, while performing adequately for the higher resourced ones. We attempt to provide insights into both the quality of the anchors, and the performance for low-shot cross-lingual transfer to better understand this performance gap. We make the extracted anchors and the modified train and test sets available for future research at https://github.com/pranaydeeps/Vyaapak %U https://aclanthology.org/2022.sigul-1.23 %P 176-184