Deep Exploration of Cross-Lingual Zero-Shot Generalization in Instruction Tuning

Janghoon Han, Changho Lee, Joongbo Shin, Stanley Jungkyu Choi, Honglak Lee, Kyunghoon Bae


Abstract
Instruction tuning has emerged as a powerful technique, significantly boosting zero-shot performance on unseen tasks. While recent work has explored cross-lingual generalization by applying instruction tuning to multilingual models, previous studies have primarily focused on English, with a limited exploration of non-English tasks. For in-depth exploration of cross-lingual generalization in instruction tuning, we perform instruction tuning individually for two distinct language meta-datasets. Subsequently, we assess the performance on unseen tasks in the language different from the one used for training. To facilitate this investigation, we introduce a novel non-English meta-dataset named “KORANI” (Korean Natural Instruction), comprising 51 Korean benchmarks. Moreover, we design cross-lingual templates to mitigate discrepancies in language and instruction-format of the template between training and inference within the cross-lingual setting. Our experiments reveal consistent improvements through cross-lingual generalization in both English and Korean, outperforming baseline by average scores of 20.7% and 13.6%, respectively. Remarkably, these enhancements are comparable to those achieved by mono-lingual instruction tuning and even surpass them in some tasks. The result underscores the significance of relevant data acquisition across languages over linguistic congruence with unseen tasks during instruction tuning.
Anthology ID:
2024.findings-acl.912
Volume:
Findings of the Association for Computational Linguistics: ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
15436–15452
Language:
URL:
https://aclanthology.org/2024.findings-acl.912
DOI:
10.18653/v1/2024.findings-acl.912
Bibkey:
Cite (ACL):
Janghoon Han, Changho Lee, Joongbo Shin, Stanley Jungkyu Choi, Honglak Lee, and Kyunghoon Bae. 2024. Deep Exploration of Cross-Lingual Zero-Shot Generalization in Instruction Tuning. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15436–15452, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Deep Exploration of Cross-Lingual Zero-Shot Generalization in Instruction Tuning (Han et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-acl.912.pdf