Do Charge Prediction Models Learn Legal Theory?

Zhenwei An, Quzhe Huang, Cong Jiang, Yansong Feng, Dongyan Zhao


Abstract
The charge prediction task aims to predict the charge for a case given its fact description. Recent models have already achieved impressive accuracy in this task, however, little is understood about the mechanisms they use to perform the judgment. For practical applications, a charge prediction model should conform to the certain legal theory in civil law countries, as under the framework of civil law, all cases are judged according to certain local legal theories. In China, for example, nearly all criminal judges make decisions based on the Four Elements Theory (FET).In this paper, we argue that trustworthy charge prediction models should take legal theories into consideration, and standing on prior studies in model interpretation, we propose three principles for trustworthy models should follow in this task, which are sensitive, selective, and presumption of innocence. We further design a new framework to evaluate whether existing charge prediction models learn legal theories. Our findings indicate that, while existing charge prediction models meet the selective principle on a benchmark dataset, most of them are still not sensitive enough and do not satisfy the presumption of innocence. Our code and dataset are released at https://github.com/ZhenweiAn/EXP_LJP.
Anthology ID:
2022.findings-emnlp.275
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2022
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Editors:
Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3757–3768
Language:
URL:
https://aclanthology.org/2022.findings-emnlp.275
DOI:
10.18653/v1/2022.findings-emnlp.275
Bibkey:
Cite (ACL):
Zhenwei An, Quzhe Huang, Cong Jiang, Yansong Feng, and Dongyan Zhao. 2022. Do Charge Prediction Models Learn Legal Theory?. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3757–3768, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
Do Charge Prediction Models Learn Legal Theory? (An et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-emnlp.275.pdf
Video:
 https://aclanthology.org/2022.findings-emnlp.275.mp4