Automated Refugee Case Analysis: A NLP Pipeline for Supporting Legal Practitioners

Claire Barale, Michael Rovatsos, Nehal Bhuta


Abstract
In this paper, we introduce an end-to-end pipeline for retrieving, processing, and extracting targeted information from legal cases. We investigate an under-studied legal domain with a case study on refugee law Canada. Searching case law for past similar cases is a key part of legal work for both lawyers and judges, the potential end-users of our prototype. While traditional named-entity recognition labels such as dates are meaningful information in law, we propose to extend existing models and retrieve a total of 19 categories of items from refugee cases. After creating a novel data set of cases, we perform information extraction based on state-of-the-art neural named-entity recognition (NER). We test different architectures including two transformer models, using contextual and non-contextual embeddings, and compare general purpose versus domain-specific pre-training. The results demonstrate that models pre-trained on legal data perform best despite their smaller size, suggesting that domain-matching had a larger effect than network architecture. We achieve a F1- score superior to 90% on five of the targeted categories and superior to 80% on an additional 4 categories.
Anthology ID:
2023.findings-acl.187
Volume:
Findings of the Association for Computational Linguistics: ACL 2023
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2992–3005
Language:
URL:
https://aclanthology.org/2023.findings-acl.187
DOI:
10.18653/v1/2023.findings-acl.187
Bibkey:
Cite (ACL):
Claire Barale, Michael Rovatsos, and Nehal Bhuta. 2023. Automated Refugee Case Analysis: A NLP Pipeline for Supporting Legal Practitioners. In Findings of the Association for Computational Linguistics: ACL 2023, pages 2992–3005, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Automated Refugee Case Analysis: A NLP Pipeline for Supporting Legal Practitioners (Barale et al., Findings 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.findings-acl.187.pdf