Augmenting Legal Decision Support Systems with LLM-based NLI for Analyzing Social Media Evidence

Ram Mohan Rao Kadiyala, Siddartha Pullakhandam, Kanwal Mehreen, Subhasya Tippareddy, Ashay Srivastava


Abstract
This paper presents our system description and error analysis of our entry for NLLP 2024 shared task on Legal Natural Language Inference (L-NLI). The task required classifying these relationships as entailed, contradicted, or neutral, indicating any association between the review and the complaint. Our system emerged as the winning submission, significantly outperforming other entries with a substantial margin and demonstrating the effectiveness of our approach in legal text analysis. We provide a detailed analysis of the strengths and limitations of each model and approach tested, along with a thorough error analysis and suggestions for future improvements. This paper aims to contribute to the growing field of legal NLP by offering insights into advanced techniques for natural language inference in legal contexts, making it accessible to both experts and newcomers in the field.
Anthology ID:
2024.nllp-1.26
Volume:
Proceedings of the Natural Legal Language Processing Workshop 2024
Month:
November
Year:
2024
Address:
Miami, FL, USA
Editors:
Nikolaos Aletras, Ilias Chalkidis, Leslie Barrett, Cătălina Goanță, Daniel Preoțiuc-Pietro, Gerasimos Spanakis
Venue:
NLLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
318–325
Language:
URL:
https://aclanthology.org/2024.nllp-1.26
DOI:
Bibkey:
Cite (ACL):
Ram Mohan Rao Kadiyala, Siddartha Pullakhandam, Kanwal Mehreen, Subhasya Tippareddy, and Ashay Srivastava. 2024. Augmenting Legal Decision Support Systems with LLM-based NLI for Analyzing Social Media Evidence. In Proceedings of the Natural Legal Language Processing Workshop 2024, pages 318–325, Miami, FL, USA. Association for Computational Linguistics.
Cite (Informal):
Augmenting Legal Decision Support Systems with LLM-based NLI for Analyzing Social Media Evidence (Kadiyala et al., NLLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.nllp-1.26.pdf