LegalPincite: Multi-level Legal Information Retrieval Dataset

Theresia Veronika Rampisela, Henrik Palmer Olsen, Giovanni Colavizza


Abstract
A common task in legal Information Retrieval (IR) is to find relevant legal sources from case-law collections. While legal practice often requires pinpoint citations (pincites) to specific case paragraphs, most existing public legal IR datasets lack paragraph-level citation annotations. Yet, publicly available datasets with such information contain data leakage in the query text and exclude paragraphs that are neither citing nor cited from the corpora, creating an unrealistic and oversimplified retrieval setting, potentially leading to inflated performance. To address these limitations, we contribute a large-scale legal IR dataset constructed from Court of Justice of the European Union (CJEU) judgments. The dataset contains: (i) masked case/paragraph queries, with removed citation information; (ii) a corpus that includes all paragraphs; and (iii) case- and paragraph-level ground truth citations, with partial human expert validation. Our dataset supports both the development and rigorous evaluation of legal IR methods, at multiple query-document levels (case-to-case, paragraph-to-case, and paragraph-to-paragraph retrieval). Link to dataset and code: https://huggingface.co/datasets/theresiavr/legalpincite
Anthology ID:
2026.nllp-1.5
Volume:
Proceedings of the Natural Legal Language Processing Workshop 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Nikolaos Aletras, Ilias Chalkidis, Leslie Barrett, Cătălina Goanță, Daniel Preoțiuc-Pietro, Gerasimos Spanakis, Shanshan Xu
Venues:
NLLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
29–41
Language:
URL:
https://aclanthology.org/2026.nllp-1.5/
DOI:
Bibkey:
Cite (ACL):
Theresia Veronika Rampisela, Henrik Palmer Olsen, and Giovanni Colavizza. 2026. LegalPincite: Multi-level Legal Information Retrieval Dataset. In Proceedings of the Natural Legal Language Processing Workshop 2026, pages 29–41, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
LegalPincite: Multi-level Legal Information Retrieval Dataset (Rampisela et al., NLLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.nllp-1.5.pdf