Identifying Token-Level Dialectal Features in Social Media

Jeremy Barnes, Samia Touileb, Petter Mæhlum, Pierre Lison


Abstract
Dialectal variation is present in many human languages and is attracting a growing interest in NLP. Most previous work concentrated on either (1) classifying dialectal varieties at the document or sentence level or (2) performing standard NLP tasks on dialectal data. In this paper, we propose the novel task of token-level dialectal feature prediction. We present a set of fine-grained annotation guidelines for Norwegian dialects, expand a corpus of dialectal tweets, and manually annotate them using the introduced guidelines. Furthermore, to evaluate the learnability of our task, we conduct labeling experiments using a collection of baselines, weakly supervised and supervised sequence labeling models. The obtained results show that, despite the difficulty of the task and the scarcity of training data, many dialectal features can be predicted with reasonably high accuracy.
Anthology ID:
2023.nodalida-1.16
Volume:
Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)
Month:
May
Year:
2023
Address:
Tórshavn, Faroe Islands
Editors:
Tanel Alumäe, Mark Fishel
Venue:
NoDaLiDa
SIG:
Publisher:
University of Tartu Library
Note:
Pages:
146–158
Language:
URL:
https://aclanthology.org/2023.nodalida-1.16
DOI:
Bibkey:
Cite (ACL):
Jeremy Barnes, Samia Touileb, Petter Mæhlum, and Pierre Lison. 2023. Identifying Token-Level Dialectal Features in Social Media. In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pages 146–158, Tórshavn, Faroe Islands. University of Tartu Library.
Cite (Informal):
Identifying Token-Level Dialectal Features in Social Media (Barnes et al., NoDaLiDa 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.nodalida-1.16.pdf