Damian Curran
Author directory2026
A Dataset of Motion-Response Text Pairs from US District Court Cases for Legal Judgment Prediction
Damian Curran | Eduard Hovy | Lea Frermann
Proceedings of the Natural Legal Language Processing Workshop 2026
Damian Curran | Eduard Hovy | Lea Frermann
Proceedings of the Natural Legal Language Processing Workshop 2026
Many existing Legal Judgment Prediction (LJP) datasets are based on test conditions which do not resemble real-world contexts, in that both the input to the task and the predicted case outcome labels are drawn from the same document. In other words, the outcome was known at the time the input was written. By utilizing the chronological event logs of court cases in US district courts, called dockets, we create a 1.6k instance outcome prediction dataset called MOP-2026 where the text inputs predate the outcome. We describe the collection, filtering and labelling of the pairs of opposing argument text and define a new subtask of LJP called motion-outcome prediction with a ternary label set. We provide baseline results and measure the performance of a variety of LLMs on the task. While exceeding naive baselines, even the latest LLMs perform below most reported metrics in LJP, supporting recent commentary that valid test conditions greatly impact predictive performance, and that many earlier LJP systems appear to overstate predictive capability in real world settings. We close with a discussion of the inherent challenges of LJP.