Annotation Study of Japanese Judgments on Tort for Legal Judgment Prediction with Rationales

Hiroaki Yamada, Takenobu Tokunaga, Ryutaro Ohara, Keisuke Takeshita, Mihoko Sumida


Abstract
This paper describes a comprehensive annotation study on Japanese judgment documents in civil cases. We aim to build an annotated corpus designed for Legal Judgment Prediction (LJP), especially for torts. Our annotation scheme contains annotations of whether tort is accepted by judges as well as its corresponding rationales for explainability purpose. Our annotation scheme extracts decisions and rationales at character-level. Moreover, the scheme can capture the explicit causal relation between judge’s decisions and their corresponding rationales, allowing multiple decisions in a document. To obtain high-quality annotation, we developed an annotation scheme with legal experts, and confirmed its reliability by agreement studies with Krippendorff’s alpha metric. The result of the annotation study suggests the proposed annotation scheme can produce a dataset of Japanese LJP at reasonable reliability.
Anthology ID:
2022.lrec-1.83
Original:
2022.lrec-1.83v1
Version 2:
2022.lrec-1.83v2
Volume:
Proceedings of the Thirteenth Language Resources and Evaluation Conference
Month:
June
Year:
2022
Address:
Marseille, France
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
779–790
Language:
URL:
https://aclanthology.org/2022.lrec-1.83
DOI:
Bibkey:
Cite (ACL):
Hiroaki Yamada, Takenobu Tokunaga, Ryutaro Ohara, Keisuke Takeshita, and Mihoko Sumida. 2022. Annotation Study of Japanese Judgments on Tort for Legal Judgment Prediction with Rationales. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 779–790, Marseille, France. European Language Resources Association.
Cite (Informal):
Annotation Study of Japanese Judgments on Tort for Legal Judgment Prediction with Rationales (Yamada et al., LREC 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.lrec-1.83.pdf
Data
ECHRECtHR