Donors and Recipients: On Asymmetric Transfer Across Tasks and Languages with Parameter-Efficient Fine-Tuning

Kajetan Dymkiewicz, Ivan Vulić, Helen Yannakoudakis, Eilam Shapira, Roi Reichart, Anna Korhonen


Abstract
Large language models (LLMs) perform strongly across tasks and languages, yet how improvements in one task or language affect other tasks and languages remains poorly understood. We conduct a controlled LoRA fine-tuning study across multiple open-weight LLM families and scales, using a standardised grid of 11 languages and four benchmarks. We fine-tune each model on a single task–language source, then evaluate it on all other task–language target pairs to measure transfer. We decompose transfer into three regimes: (i) Matched-Task (Cross-Language), (ii) Cross-Task (Matched-Language), and (iii) Cross-Task (Cross-Language). Single-source fine-tuning yields a net positive uplift across regimes, but the gains are strongly asymmetric. Matched-Task (Cross-Language) transfer emerges as the most effective and structurally regular regime, with transfer magnitude driven principally by the identity of the target language rather than model architecture. We identify a stable coarse-grained hierarchy in which some task and language targets consistently absorb gains from diverse sources, while others remain relatively isolated. These results imply that effective fine-tuning requires accounting for donor–recipient roles to maximise downstream gains while limiting collateral degradation in other capabilities.
Anthology ID:
2026.findings-emnlp.67
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1356–1373
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.67/
DOI:
Bibkey:
Cite (ACL):
Kajetan Dymkiewicz, Ivan Vulić, Helen Yannakoudakis, Eilam Shapira, Roi Reichart, and Anna Korhonen. 2026. Donors and Recipients: On Asymmetric Transfer Across Tasks and Languages with Parameter-Efficient Fine-Tuning. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 1356–1373, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Donors and Recipients: On Asymmetric Transfer Across Tasks and Languages with Parameter-Efficient Fine-Tuning (Dymkiewicz et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.67.pdf