Tug-of-War Tuning: Capacity and Cross-Lingual Transfer in Zero-Shot NER

Tanushree Ravindra Pratap Yadav


Abstract
We investigate how the amount of fine-tuned capacity affects zero-shot cross-lingual transfer for named entity recognition (NER). Training exclusively on English WikiANN and evaluating on 14 typologically diverse target languages, we find a consistent inverted-U pattern: cross-lingual transfer is worst at both ends of the parameter-efficiency spectrum. Full fine-tuning, which updates all parameters, transfers worst overall and collapses on non-Latin-script languages, while the most parameter-efficient methods, (IA)³ and prefix tuning, also underperform. Methods with an intermediate number of trainable parameters (Low-Rank Adaptation, LoRA; Hu et al., 2022, and Houlsby adapters) transfer best, with LoRA outperforming full fine-tuning by roughly 7 F1 points on average and by over 10 points on languages such as Chinese and Japanese. We find the same inverted-U shape within LoRA alone: a sweep over ranks shows cross-lingual transfer peaking at an intermediate rank, distinct from the higher rank that maximises in-language English performance, revealing a rank–transfer tradeoff. We also examine, and do not find clean support for, the common assumption that better preservation of cross-lingual representational alignment explains PEFT’s transfer benefits: the method that best preserves a standard alignment measure transfers worst, while LoRA’s larger alignment shift coincides with its best transfer. Given known limitations of this alignment measure, we present this as a preliminary, metric-dependent observation that motivates further analysis with more robust alignment metrics, rather than a settled claim.
Anthology ID:
2026.winlp-main.1
Volume:
Proceedings of the 10th Widening NLP Workshop
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Chen Zhang, Emily Allaway, Meryem M’hamdi, Peerat Limkonchotiwat, Richard He Bai, Santosh T.Y.S.S., Surendrabikram Thapa, Yinqiao Li
Venues:
WiNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–9
Language:
URL:
https://aclanthology.org/2026.winlp-main.1/
DOI:
Bibkey:
Cite (ACL):
Tanushree Ravindra Pratap Yadav. 2026. Tug-of-War Tuning: Capacity and Cross-Lingual Transfer in Zero-Shot NER. In Proceedings of the 10th Widening NLP Workshop, pages 1–9, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Tug-of-War Tuning: Capacity and Cross-Lingual Transfer in Zero-Shot NER (Yadav, WiNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.winlp-main.1.pdf