A dual-encoding system for dialect classification

Petru Rebeja, Dan Cristea


Abstract
In this paper we present the architecture, processing pipeline and results of the ensemble model developed for Romanian Dialect Identification task. The ensemble model consists of two TF-IDF encoders and a deep learning model aimed together at classifying input samples based on the writing patterns which are specific to each of the two dialects. Although the model performs well on the training set, its performance degrades heavily on the evaluation set. The drop in performance is due to the design decision which makes the model put too much weight on presence/lack of textual marks when determining the sample label.
Anthology ID:
2020.vardial-1.20
Volume:
Proceedings of the 7th Workshop on NLP for Similar Languages, Varieties and Dialects
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Marcos Zampieri, Preslav Nakov, Nikola Ljubešić, Jörg Tiedemann, Yves Scherrer
Venue:
VarDial
SIG:
Publisher:
International Committee on Computational Linguistics (ICCL)
Note:
Pages:
212–219
Language:
URL:
https://aclanthology.org/2020.vardial-1.20
DOI:
Bibkey:
Cite (ACL):
Petru Rebeja and Dan Cristea. 2020. A dual-encoding system for dialect classification. In Proceedings of the 7th Workshop on NLP for Similar Languages, Varieties and Dialects, pages 212–219, Barcelona, Spain (Online). International Committee on Computational Linguistics (ICCL).
Cite (Informal):
A dual-encoding system for dialect classification (Rebeja & Cristea, VarDial 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.vardial-1.20.pdf
Code
 repierre/vardial2020
Data
MOROCO