When Every Token Counts: Optimal Segmentation for Low-Resource Language Models

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Bharath Raj S author Garvit Suri author Vikrant Dewangan author Raghav Sonavane author 2025-01 text Proceedings of the First Workshop on Language Models for Low-Resource Languages Hansi Hettiarachchi editor Tharindu Ranasinghe editor Paul Rayson editor Ruslan Mitkov editor Mohamed Gaber editor Damith Premasiri editor Fiona Anting Tan editor Lasitha Uyangodage editor Association for Computational Linguistics Abu Dhabi, United Arab Emirates conference publication dewangan-etal-2025-every https://aclanthology.org/2025.loreslm-1.24/ 2025-01 294 308