Joseba Fernandez de Landa
2026
SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures
Nedjma Ousidhoum | Junho Myung | Carla Perez-Almendros | Jiho Jin | Amr Keleg | Meriem Beloucif | Yi Zhou | Rodrigo Agerri | Vladimir Araujo | Naomi Baes | James Barry | Joanne Boisson | Nancy F. Chen | Christine de Kock | Aleksandra Edwards | Joseba Fernandez de Landa | Mohamed Fazli Imam | Huda Hakami | Shu-Kai Hsieh | Joseph Marvin Imperial | Roy Ka-Wei Lee | Zhengyuan Liu | Chenyang Lyu | Younes Samih | Johan Sjons | Bryan Tan | Asahi Ushio | Weihua Zheng | Alice Oh | Jose Camacho-Collados
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Nedjma Ousidhoum | Junho Myung | Carla Perez-Almendros | Jiho Jin | Amr Keleg | Meriem Beloucif | Yi Zhou | Rodrigo Agerri | Vladimir Araujo | Naomi Baes | James Barry | Joanne Boisson | Nancy F. Chen | Christine de Kock | Aleksandra Edwards | Joseba Fernandez de Landa | Mohamed Fazli Imam | Huda Hakami | Shu-Kai Hsieh | Joseph Marvin Imperial | Roy Ka-Wei Lee | Zhengyuan Liu | Chenyang Lyu | Younes Samih | Johan Sjons | Bryan Tan | Asahi Ushio | Weihua Zheng | Alice Oh | Jose Camacho-Collados
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
We present our shared task on evaluating the adaptability of LLMs and NLP systems across multiple languages and cultures. The task data consist of an extended version of our manually constructed BLEnD benchmark (Myung et al., 2024), covering more than 30 language–culture pairs, predominantly representing low-resource languages spoken across multiple continents. As the task is designed strictly for evaluation, participants were not permitted to use the data for training, fine-tuning, few-shot learning, or any other form of model modification.Our task includes two tracks: (a) Short-Answer Questions (SAQ) and (b) Multiple-Choice Questions (MCQ). Participants were required to predict labels and were allowed to submit any NLP system and adopt diverse modelling strategies, provided that the benchmark was used solely for evaluation. The task attracted more than 140 registered participants, and we received final submissions from 62 teams, along with 19 system description papers.We report the results and present an analysis of the best-performing systems and the most commonly adopted approaches. Furthermore, we discuss shared insights into open questions and challenges related to evaluation, misalignment, and methodological perspectives on model behaviour in low-resource languages and for under-represented cultures. Our data and resources are available at https://github.com/BLEnD-SemEval2026/SemEval-2026-Task-7.
2025
Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basque
Oscar Sainz | Naiara Perez | Julen Etxaniz | Joseba Fernandez de Landa | Itziar Aldabe | Iker García-Ferrero | Aimar Zabala | Ekhi Azurmendi | German Rigau | Eneko Agirre | Mikel Artetxe | Aitor Soroa
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Oscar Sainz | Naiara Perez | Julen Etxaniz | Joseba Fernandez de Landa | Itziar Aldabe | Iker García-Ferrero | Aimar Zabala | Ekhi Azurmendi | German Rigau | Eneko Agirre | Mikel Artetxe | Aitor Soroa
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Instructing language models with user intent requires large instruction datasets, which are only available for a limited set of languages. In this paper, we explore alternatives to conventional instruction adaptation pipelines in low-resource scenarios. We assume a realistic scenario for low-resource languages, where only the following are available: corpora in the target language, existing open-weight multilingual base and instructed backbone LLMs, and synthetically generated instructions sampled from the instructed backbone. We present a comprehensive set of experiments for Basque that systematically study different combinations of these components evaluated on benchmarks and human preferences from 1,680 participants. Our conclusions show that target language corpora are essential, with synthetic instructions yielding robust models, and, most importantly, that using as backbone an instruction-tuned model outperforms using a base non-instructed model. Scaling up to Llama 3.1 Instruct 70B as backbone, our model comes near frontier models of much larger sizes for Basque, without using any Basque instructions. We release code, models, instruction datasets, and human preferences to support full reproducibility in future research on low-resource language adaptation.
2024
Uncovering Social Changes of the Basque Speaking Twitter Community During COVID-19 Pandemic
Joseba Fernandez de Landa | Iker García-Ferrero | Ander Salaberria | Jon Ander Campos
Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024
Joseba Fernandez de Landa | Iker García-Ferrero | Ander Salaberria | Jon Ander Campos
Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024
The aim of this work is to study the impact of the COVID-19 pandemic on the Basque speaking Twitter community by applying Natural Language Processing unsupervised techniques. In order to carry out this study, we collected and publicly released the biggest dataset of Basque tweets containing up to 8M tweets from September 2019 to February 2021. To analyze the impact of the pandemic, the variability of the content over time was studied through quantitative and qualitative analysis of words and emojis. For the quantitative analysis, the shift at the frequency of the terms was calculated using linear regression over frequencies. On the other hand, for the qualitative analysis, word embeddings were used to study the changes in the meaning of the most significant words and emojis at different periods of the pandemic. Through this multifaceted approach, we discovered noteworthy alterations in the political inclinations exhibited by Basque users throughout the course of the pandemic.
Search
Fix author
Co-authors
- Iker García-Ferrero 2
- Rodrigo Agerri 1
- Eneko Agirre 1
- Itziar Aldabe 1
- Vladimir Araujo 1
- Mikel Artetxe 1
- Ekhi Azurmendi 1
- Naomi Baes 1
- James Barry 1
- Meriem Beloucif 1
- Joanne Boisson 1
- Jose Camacho-Collados 1
- Jon Ander Campos 1
- Nancy Chen 1
- Aleksandra Edwards 1
- Julen Etxaniz 1
- Mohamed Fazli Imam 1
- Huda Hakami 1
- Shu-Kai Hsieh 1
- Joseph Marvin Imperial 1
- Jiho Jin 1
- Amr Keleg 1
- Roy Ka-Wei Lee 1
- Zhengyuan Liu 1
- Chenyang Lyu 1
- Junho Myung 1
- Alice Oh 1
- Nedjma Ousidhoum 1
- Carla Perez-Almendros 1
- Naiara Pérez 1
- German Rigau 1
- Oscar Sainz 1
- Ander Salaberria 1
- Younes Samih 1
- Johan Sjons 1
- Aitor Soroa 1
- Bryan Tan 1
- Asahi Ushio 1
- Aimar Zabala 1
- Weihua Zheng 1
- Yi Zhou 1
- Christine de Kock 1