UO-LouTAL at SemEval-2023 Task 6: Lightweight Systems for Legal Processing

Sébastien Bosch, Louis Estève, Joanne Loo, Anne-Lyse Minard


Abstract
This paper presents the work produced by students of the University of Orlans Masters in Natural Language Processing program by way of participating in SemEval Task 6, LegalEval, which aims to enhance the capabilities of legal professionals through automated systems. Two out of the three sub-tasks available – Rhetorical Role prediction (RR) and Legal Named Entity Recognition (L-NER) – were tackled, with the express intent of developing lightweight and interpretable systems. For the L-NER sub-task, a CRF model was trained, augmented with post-processing rules for some named entity types. A macro F1 score of 0.74 was obtained on the DEV set, and 0.64 on the evaluation set. As for the RR sub-task, two sentence classification systems were built: one based on the Bag-of-Words technique with L-NER system output integrated, the other using a sentence-transformer approach. Rule-based post-processing then converted the results of the sentence classification systems into RR predictions. The better-performing Bag-of-Words system obtained a macro F1 score of 0.49 on the DEV set and 0.57 on the evaluation set.
Anthology ID:
2023.semeval-1.195
Volume:
Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Atul Kr. Ojha, A. Seza Doğruöz, Giovanni Da San Martino, Harish Tayyar Madabushi, Ritesh Kumar, Elisa Sartori
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
1412–1420
Language:
URL:
https://aclanthology.org/2023.semeval-1.195
DOI:
10.18653/v1/2023.semeval-1.195
Bibkey:
Cite (ACL):
Sébastien Bosch, Louis Estève, Joanne Loo, and Anne-Lyse Minard. 2023. UO-LouTAL at SemEval-2023 Task 6: Lightweight Systems for Legal Processing. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1412–1420, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
UO-LouTAL at SemEval-2023 Task 6: Lightweight Systems for Legal Processing (Bosch et al., SemEval 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.semeval-1.195.pdf