Evidence-Grounded Contract Clause Classification with Small Language Models

Dina Mohamed Ali, Hager Hamed Abd Elrady Allam, Hayam Abdelbaset Rezk, Rahatara Ferdousi, Mohammed Anber AbdElhamed


Abstract
Commercial contract clauses exhibit substantial linguistic variation and semantic overlap, making fine-grained classification challenging for small language models (SLMs). We investigate whether retrieving semantically similar labeled clauses can provide task-specific evidence at inference time and improve classification without modifying the underlying SLM. Using the Contract Understanding Atticus Dataset (CUAD), we formulate clause classification as a retrieval-conditioned prediction task in which relevant training clauses are retrieved and provided as in-context demonstrations to a locally deployed SLM. We evaluate prompting-only, retrieval-only, and retrieval-conditioned configurations on the same contract-disjoint held-out set of 1,679 clauses. The proposed retrieval-conditioned SLM achieves 66.41% accuracy and 51.72% macro-F1, compared with 35.80% and 31.41% for the zero-shot SLM and 50.98% and 40.93% for the random few-shot SLM, respectively. Retrieval-only nearest-neighbor baselines achieve 47.48–47.82% macro-F1, while conditioning the SLM on the same retrieved evidence increases macro-F1 to 51.72%. These results show that retrieval provides substantial task-specific context and that SLM conditioning provides additional improvement beyond direct nearest-neighbor label assignment, without task-specific parameter fine-tuning. The findings support retrieval-conditioned SLMs as a promising direction for evidence-grounded and locally deployable legal NLP
Anthology ID:
2026.nllp-1.7
Volume:
Proceedings of the Natural Legal Language Processing Workshop 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Nikolaos Aletras, Ilias Chalkidis, Leslie Barrett, Cătălina Goanță, Daniel Preoțiuc-Pietro, Gerasimos Spanakis, Shanshan Xu
Venues:
NLLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
55–63
Language:
URL:
https://aclanthology.org/2026.nllp-1.7/
DOI:
Bibkey:
Cite (ACL):
Dina Mohamed Ali, Hager Hamed Abd Elrady Allam, Hayam Abdelbaset Rezk, Rahatara Ferdousi, and Mohammed Anber AbdElhamed. 2026. Evidence-Grounded Contract Clause Classification with Small Language Models. In Proceedings of the Natural Legal Language Processing Workshop 2026, pages 55–63, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Evidence-Grounded Contract Clause Classification with Small Language Models (Ali et al., NLLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.nllp-1.7.pdf