Reliable Baselines for Sentiment Analysis in Resource-Limited Languages: The Serbian Movie Review Dataset

Vuk Batanović, Boško Nikolić, Milan Milosavljević


Abstract
Collecting data for sentiment analysis in resource-limited languages carries a significant risk of sample selection bias, since the small quantities of available data are most likely not representative of the whole population. Ignoring this bias leads to less robust machine learning classifiers and less reliable evaluation results. In this paper we present a dataset balancing algorithm that minimizes the sample selection bias by eliminating irrelevant systematic differences between the sentiment classes. We prove its superiority over the random sampling method and we use it to create the Serbian movie review dataset ― SerbMR ― the first balanced and topically uniform sentiment analysis dataset in Serbian. In addition, we propose an incremental way of finding the optimal combination of simple text processing options and machine learning features for sentiment classification. Several popular classifiers are used in conjunction with this evaluation approach in order to establish strong but reliable baselines for sentiment analysis in Serbian.
Anthology ID:
L16-1427
Volume:
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
Month:
May
Year:
2016
Address:
Portorož, Slovenia
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
2688–2696
Language:
URL:
https://aclanthology.org/L16-1427
DOI:
Bibkey:
Cite (ACL):
Vuk Batanović, Boško Nikolić, and Milan Milosavljević. 2016. Reliable Baselines for Sentiment Analysis in Resource-Limited Languages: The Serbian Movie Review Dataset. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 2688–2696, Portorož, Slovenia. European Language Resources Association (ELRA).
Cite (Informal):
Reliable Baselines for Sentiment Analysis in Resource-Limited Languages: The Serbian Movie Review Dataset (Batanović et al., LREC 2016)
Copy Citation:
PDF:
https://aclanthology.org/L16-1427.pdf