jurBERT: A Romanian BERT Model for Legal Judgement Prediction

Mihai Masala, Radu Cristian Alexandru Iacob, Ana Sabina Uban, Marina Cidota, Horia Velicu, Traian Rebedea, Marius Popescu


Abstract
Transformer-based models have become the de facto standard in the field of Natural Language Processing (NLP). By leveraging large unlabeled text corpora, they enable efficient transfer learning leading to state-of-the-art results on numerous NLP tasks. Nevertheless, for low resource languages and highly specialized tasks, transformer models tend to lag behind more classical approaches (e.g. SVM, LSTM) due to the lack of aforementioned corpora. In this paper we focus on the legal domain and we introduce a Romanian BERT model pre-trained on a large specialized corpus. Our model outperforms several strong baselines for legal judgement prediction on two different corpora consisting of cases from trials involving banks in Romania.
Anthology ID:
2021.nllp-1.8
Volume:
Proceedings of the Natural Legal Language Processing Workshop 2021
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Venue:
NLLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
86–94
Language:
URL:
https://aclanthology.org/2021.nllp-1.8
DOI:
10.18653/v1/2021.nllp-1.8
Bibkey:
Cite (ACL):
Mihai Masala, Radu Cristian Alexandru Iacob, Ana Sabina Uban, Marina Cidota, Horia Velicu, Traian Rebedea, and Marius Popescu. 2021. jurBERT: A Romanian BERT Model for Legal Judgement Prediction. In Proceedings of the Natural Legal Language Processing Workshop 2021, pages 86–94, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
jurBERT: A Romanian BERT Model for Legal Judgement Prediction (Masala et al., NLLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.nllp-1.8.pdf