Does Fairness Survive Model Adaptation in Legal Judgment Prediction?

Sarah Kamoun, Parth Patel, Faezeh Ensan


Abstract
Large language models are increasingly used for Legal Judgment Prediction (LJP), raising concerns about demographic bias inherited from pre-training corpora and historical legal records. We investigate how fairness robustness changes under model adaptation in LJP through a controlled counterfactual evaluation across gender, ethnicity, and intersectional demographic groups. Using 23 demographic variants over encoder-based models and generative LLMs, we evaluate demographic sensitivity, counterfactual instability, and bias propagation under zero-shot, few-shot, and fine-tuned settings. Experiments show that model adaptation substantially affects fairness. Fine-tuning often improves predictive performance while simultaneously increasing demographic sensitivity and propagating biased supervision signals into downstream judicial predictions, whereas few-shot prompting generally produces smaller propagation effects. We further show that legal-domain specialization does not reliably guarantee fairness: legal-domain encoders span both highly robust and highly sensitive behavior after fine-tuning, and a legal-domain LLM is highly sensitive under few-shot prompting yet comparatively robust after fine-tuning under counterfactual perturbation.
Anthology ID:
2026.aacl-long.254
Volume:
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venues:
AACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5653–5667
Language:
URL:
https://aclanthology.org/2026.aacl-long.254/
DOI:
Bibkey:
Cite (ACL):
Sarah Kamoun, Parth Patel, and Faezeh Ensan. 2026. Does Fairness Survive Model Adaptation in Legal Judgment Prediction?. In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5653–5667, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
Does Fairness Survive Model Adaptation in Legal Judgment Prediction? (Kamoun et al., AACL-IJCNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aacl-long.254.pdf