Iñaki Lacunza Castilla
2024
Community OSCAR: A Community Effort for Multilingual Web Data
Manuel Brack
|
Malte Ostendorff
|
Pedro Ortiz Suarez
|
José Javier Saiz
|
Iñaki Lacunza Castilla
|
Jorge Palomar-Giner
|
Alexander Shvets
|
Patrick Schramowski
|
Georg Rehm
|
Marta Villegas
|
Kristian Kersting
Proceedings of the Fourth Workshop on Multilingual Representation Learning (MRL 2024)
The development of large language models (LLMs) relies heavily on extensive, high-quality datasets. Publicly available datasets focus predominantly on English, leaving other language communities behind. To address this issue, we introduce Community OSCAR, a multilingual dataset initiative designed to address the gap between English and non-English data availability. Through a collective effort, Community OSCAR covers over 150 languages with 45 billion documents, totaling over 345 TiB of data. Initial results indicate that Community OSCAR provides valuable raw data for training LLMs and enhancing the performance of multilingual models. This work aims to contribute to the ongoing advancements in multilingual NLP and to support a more inclusive AI ecosystem by making high-quality, multilingual data more accessible to those working with low-resource languages.
Search
Co-authors
- Manuel Brack 1
- Malte Ostendorff 1
- Pedro Ortiz Suarez 1
- José Javier Saiz 1
- Jorge Palomar-Giner 1
- show all...
Venues
- mrl1