Challenges for Information Extraction from Dialogue in Criminal Law

Jenny Hong, Catalin Voss, Christopher Manning


Abstract
Information extraction and question answering have the potential to introduce a new paradigm for how machine learning is applied to criminal law. Existing approaches generally use tabular data for predictive metrics. An alternative approach is needed for matters of equitable justice, where individuals are judged on a case-by-case basis, in a process involving verbal or written discussion and interpretation of case factors. Such discussions are individualized, but they nonetheless rely on underlying facts. Information extraction can play an important role in surfacing these facts, which are still important to understand. We analyze unsupervised, weakly supervised, and pre-trained models’ ability to extract such factual information from the free-form dialogue of California parole hearings. With a few exceptions, most F1 scores are below 0.85. We use this opportunity to highlight some opportunities for further research for information extraction and question answering. We encourage new developments in NLP to enable analysis and review of legal cases to be done in a post-hoc, not predictive, manner.
Anthology ID:
2021.nlp4posimpact-1.8
Volume:
Proceedings of the 1st Workshop on NLP for Positive Impact
Month:
August
Year:
2021
Address:
Online
Editors:
Anjalie Field, Shrimai Prabhumoye, Maarten Sap, Zhijing Jin, Jieyu Zhao, Chris Brockett
Venue:
NLP4PI
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
71–81
Language:
URL:
https://aclanthology.org/2021.nlp4posimpact-1.8
DOI:
10.18653/v1/2021.nlp4posimpact-1.8
Bibkey:
Cite (ACL):
Jenny Hong, Catalin Voss, and Christopher Manning. 2021. Challenges for Information Extraction from Dialogue in Criminal Law. In Proceedings of the 1st Workshop on NLP for Positive Impact, pages 71–81, Online. Association for Computational Linguistics.
Cite (Informal):
Challenges for Information Extraction from Dialogue in Criminal Law (Hong et al., NLP4PI 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.nlp4posimpact-1.8.pdf
Data
CoQASQuAD