Annotating Norwegian language varieties on Twitter for Part-of-speech

Petter Mæhlum, Andre Kåsen, Samia Touileb, Jeremy Barnes


Abstract
Norwegian Twitter data poses an interesting challenge for Natural Language Processing (NLP) tasks. These texts are difficult for models trained on standardized text in one of the two Norwegian written forms (Bokmål and Nynorsk), as they contain both the typical variation of social media text, as well as a large amount of dialectal variety. In this paper we present a novel Norwegian Twitter dataset annotated with POS-tags. We show that models trained on Universal Dependency (UD) data perform worse when evaluated against this dataset, and that models trained on Bokmål generally perform better than those trained on Nynorsk. We also see that performance on dialectal tweets is comparable to the written standards for some models. Finally we perform a detailed analysis of the errors that models commonly make on this data.
Anthology ID:
2022.vardial-1.7
Volume:
Proceedings of the Ninth Workshop on NLP for Similar Languages, Varieties and Dialects
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Editors:
Yves Scherrer, Tommi Jauhiainen, Nikola Ljubešić, Preslav Nakov, Jörg Tiedemann, Marcos Zampieri
Venue:
VarDial
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
64–69
Language:
URL:
https://aclanthology.org/2022.vardial-1.7
DOI:
Bibkey:
Cite (ACL):
Petter Mæhlum, Andre Kåsen, Samia Touileb, and Jeremy Barnes. 2022. Annotating Norwegian language varieties on Twitter for Part-of-speech. In Proceedings of the Ninth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 64–69, Gyeongju, Republic of Korea. Association for Computational Linguistics.
Cite (Informal):
Annotating Norwegian language varieties on Twitter for Part-of-speech (Mæhlum et al., VarDial 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.vardial-1.7.pdf