Damian Curran

Author directory

2026

Many existing Legal Judgment Prediction (LJP) datasets are based on test conditions which do not resemble real-world contexts, in that both the input to the task and the predicted case outcome labels are drawn from the same document. In other words, the outcome was known at the time the input was written. By utilizing the chronological event logs of court cases in US district courts, called dockets, we create a 1.6k instance outcome prediction dataset called MOP-2026 where the text inputs predate the outcome. We describe the collection, filtering and labelling of the pairs of opposing argument text and define a new subtask of LJP called motion-outcome prediction with a ternary label set. We provide baseline results and measure the performance of a variety of LLMs on the task. While exceeding naive baselines, even the latest LLMs perform below most reported metrics in LJP, supporting recent commentary that valid test conditions greatly impact predictive performance, and that many earlier LJP systems appear to overstate predictive capability in real world settings. We close with a discussion of the inherent challenges of LJP.