OblEx26: A Multi-Jurisdictional Benchmark for Structured Obligation Extraction from Anti-Money Laundering Regulations

Philip Moore, Giovanni Bandi, Tim Luka Horstmann, Russell Moore, Robert H. Wardrop, Paula Buttery


Abstract
Anti-money laundering and counter-terrorism financing (AML/CFT) regulations impose complex obligations on financial institutions. Auditing these obligations requires thousands of hours of manual review by legal experts, yet automated structured obligation extraction remains underexplored. In this paper we: 1) Introduce OblEx26, the first multi-jurisdictional benchmarking dataset designed for evaluating structured obligation extraction from regulatory documents. The dataset contains 48 expert-annotated documents spanning 14 jurisdictions, consisting of 43 documents in English and 5 in Brazilian Portuguese and Mandarin. Grounded in Hohfeldian legal theory and deontic logic, each obligation is decomposed into six components "Holder", "Type", "Beneficiary", "Antecedent", "Consequent", "Source / Legal Channel" and is anchored to exact quotes, enabling verifiable provenance; 2) Compare four open-weight methods for structured obligation extraction that can run privately on consumer-grade hardware: token-classification encoders; a joint classification and obligation-linking architecture; zero-shot with LLMs; and fine-tuned SLMs. Our fine-tuned SLM Qwen3.5-27B achieves a soft-match F1 score of 0.728, with 0.991 exact quote anchoring on English language data, and 0.659 F1 with 1.000 anchoring when used zero-shot on the cross-lingual data. Fine-tuned SLMs substantially outperform zero-shot LLMs and encoder-based approaches, establishing a strong baseline for automated legal compliance. OblEx26 evaluation data is publicly released.
Anthology ID:
2026.nllp-1.4
Volume:
Proceedings of the Natural Legal Language Processing Workshop 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Nikolaos Aletras, Ilias Chalkidis, Leslie Barrett, Cătălina Goanță, Daniel Preoțiuc-Pietro, Gerasimos Spanakis, Shanshan Xu
Venues:
NLLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
14–28
Language:
URL:
https://aclanthology.org/2026.nllp-1.4/
DOI:
Bibkey:
Cite (ACL):
Philip Moore, Giovanni Bandi, Tim Luka Horstmann, Russell Moore, Robert H. Wardrop, and Paula Buttery. 2026. OblEx26: A Multi-Jurisdictional Benchmark for Structured Obligation Extraction from Anti-Money Laundering Regulations. In Proceedings of the Natural Legal Language Processing Workshop 2026, pages 14–28, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
OblEx26: A Multi-Jurisdictional Benchmark for Structured Obligation Extraction from Anti-Money Laundering Regulations (Moore et al., NLLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.nllp-1.4.pdf