Enhancing Contract Negotiations with LLM-Based Legal Document Comparison

Savinay Narendra, Kaushal Shetty, Adwait Ratnaparkhi


Abstract
We present a large language model (LLM) based approach for comparing legal contracts with their corresponding template documents. Legal professionals use commonly observed deviations between templates and contracts to help with contract negotiations, and also to refine the template documents. Our comparison approach, based on the well-studied natural language inference (NLI) task, first splits a template into key concepts and then uses LLMs to decide if the concepts are entailed by the contract document. We also repeat this procedure in the opposite direction - contract clauses are tested for entailment against the template clause to see if they contain additional information. The non-entailed concepts are labelled, organized and filtered by frequency, and placed into a clause library, which is used to suggest changes to the template documents. We first show that our LLM-based approach outperforms all previous work on a publicly available dataset designed for NLI in the legal domain. We then apply it to a private real-world legal dataset, achieve an accuracy of 96.46%. Our approach is the first in the literature to produce a natural language comparison between legal contracts and their template documents.
Anthology ID:
2024.nllp-1.11
Volume:
Proceedings of the Natural Legal Language Processing Workshop 2024
Month:
November
Year:
2024
Address:
Miami, FL, USA
Editors:
Nikolaos Aletras, Ilias Chalkidis, Leslie Barrett, Cătălina Goanță, Daniel Preoțiuc-Pietro, Gerasimos Spanakis
Venue:
NLLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
143–153
Language:
URL:
https://aclanthology.org/2024.nllp-1.11
DOI:
Bibkey:
Cite (ACL):
Savinay Narendra, Kaushal Shetty, and Adwait Ratnaparkhi. 2024. Enhancing Contract Negotiations with LLM-Based Legal Document Comparison. In Proceedings of the Natural Legal Language Processing Workshop 2024, pages 143–153, Miami, FL, USA. Association for Computational Linguistics.
Cite (Informal):
Enhancing Contract Negotiations with LLM-Based Legal Document Comparison (Narendra et al., NLLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.nllp-1.11.pdf