Exploiting Domain-Specific Knowledge for Judgment Prediction Is No Panacea

Olivier Salaün, Philippe Langlais, Karim Benyekhlef


Abstract
Legal judgment prediction (LJP) usually consists in a text classification task aimed at predicting the verdict on the basis of the fact description. The literature shows that the use of articles as input features helps improve the classification performance. In this work, we designed a verdict prediction task based on landlord-tenant disputes and we applied BERT-based models to which we fed different article-based features. Although the results obtained are consistent with the literature, the improvements with the articles are mostly obtained with the most frequent labels, suggesting that pre-trained and fine-tuned transformer-based models are not scalable as is for legal reasoning in real life scenarios as they would only excel in accurately predicting the most recurrent verdicts to the detriment of other legal outcomes.
Anthology ID:
2021.ranlp-1.139
Volume:
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)
Month:
September
Year:
2021
Address:
Held Online
Editors:
Ruslan Mitkov, Galia Angelova
Venue:
RANLP
SIG:
Publisher:
INCOMA Ltd.
Note:
Pages:
1234–1243
Language:
URL:
https://aclanthology.org/2021.ranlp-1.139
DOI:
Bibkey:
Cite (ACL):
Olivier Salaün, Philippe Langlais, and Karim Benyekhlef. 2021. Exploiting Domain-Specific Knowledge for Judgment Prediction Is No Panacea. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1234–1243, Held Online. INCOMA Ltd..
Cite (Informal):
Exploiting Domain-Specific Knowledge for Judgment Prediction Is No Panacea (Salaün et al., RANLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.ranlp-1.139.pdf