As large language models (LLMs) increasingly support sensitive decision-making, understanding their vulnerability to bias becomes critical for maintaining system integrity. However, we lack reliable methods to measure how easily these systems can be compromised through biased training data. We present the first approach to quantifying “imitation thresholds“, defined as the minimum number of training examples needed to significantly shift decision patterns in LLMs applied to legal decision-making. Our experiments span four models across two size categories, 24 judicial personas, and four legal domains, revealing distinct patterns of susceptibility and resistance to persona-driven biases that persist across model architectures and scales. By incrementally fine-tuning LLMs with persona-specific text, we identify thresholds at which model predictions diverge meaningfully from control decision patterns. Our findings show that as few as 80 training examples can reliably alter model behavior, with significant variation across domains and influencer persona types. We further find that susceptibility scales with model capacity, with larger models showing substantially stronger effects, particularly for negative influence. Our work provides a foundational understanding of bias propagation mechanisms in LLMs. We publicly release our datasets and code to support future research in this area.

In this study, we focus on two main tasks, the first for detecting legal violations within unstructured textual data, and the second for associating these violations with potentially affected individuals. We constructed two datasets using Large Language Models (LLMs) which were subsequently validated by domain expert annotators. Both tasks were designed specifically for the context of class-action cases. The experimental design incorporated fine-tuning models from the BERT family and open-source LLMs, and conducting few-shot experiments using closed-source LLMs. Our results, with an F1-score of 62.69% (violation identification) and 81.02% (associating victims), show that our datasets and setups can be used for both tasks. Finally, we publicly release the datasets and the code used for the experiments in order to advance further research in the area of legal natural language processing (NLP).

The research field of Legal Natural Language Processing (NLP) has been very active recently, with Legal Judgment Prediction (LJP) becoming one of the most extensively studied tasks. To date, most publicly released LJP datasets originate from countries with civil law. In this work, we release, for the first time, a challenging LJP dataset focused on class action cases in the US. It is the first dataset in the common law system that focuses on the harder and more realistic task involving the complaints as input instead of the often used facts summary written by the court. Additionally, we study the difficulty of the task by collecting expert human predictions, showing that even human experts can only reach 53% accuracy on this dataset. Our Longformer model clearly outperforms the human baseline (63%), despite only considering the first 2,048 tokens. Furthermore, we perform a detailed error analysis and find that the Longformer model is significantly better calibrated than the human experts. Finally, we publicly release the dataset and the code used for the experiments.