Avrile Floro
Author directory2026
Where Experts Disagree, Models Fail: Detecting Implicit Legal Citations in French Court Decisions
Avrile Floro | Tamara Dhorasoo | Soline Pellez | Nils Holzenberger
Proceedings of the Natural Legal Language Processing Workshop 2026
Avrile Floro | Tamara Dhorasoo | Soline Pellez | Nils Holzenberger
Proceedings of the Natural Legal Language Processing Workshop 2026
Applying computational methods to law at scale requires separating genuine legal reasoning from surface similarity. We study this through a concrete task: detecting *implicit* citations of the French Civil Code, where a court applies a statutory rule without naming it (a *post-hoc* question about the reasoning a court actually used). We release a benchmark of 1,015 passage–article pairs annotated by three legal experts. Our central finding is that their disagreement is itself informative: the third of cases the experts dispute are where models fail. Our best ensemble reaches an F1 score of 0.70 overall. Yet, two-thirds of its false positives fall on those disputed cases, a concentration that holds across all ten models we evaluate. Expert disagreement thus signals uncertainty in the task and in its gold label, which aggregate metrics hide. This should not block useful tools, however: reframed as top-k ranking with multi-model consensus, the same signals reach 76% precision for the top-200 candidates of the benchmark without supervision.
Cultural Binding Heads in Language Models
Avrile Floro | Luca Benedetto
Proceedings of the 9th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP
Avrile Floro | Luca Benedetto
Proceedings of the 9th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP
LLMs often default to equal treatment across cultural groups, even though context warrants differentiation: this is a lack of *difference awareness*. Using mechanistic interpretability and a factorial design on the N4 cultural appropriation benchmark from Wang et al. (2025), we identify 2–3 mid-layer attention heads per model that contribute causally to cultural binding across eight models (base and instruct versions of four architectures). Cultural binding is the process of associating a cultural item with its related identity. Knockout of the identity→item edges on these heads lowers the binding strength by 9–23%. The identified heads transfer from instruct to base models, suggesting that cultural binding is created during pre-training. An 𝛼-scaling shows a graded dose–response. Moderate amplification steering at generation (𝛼 = 2–3) increases cultural differentiation accuracy by 1–3 pp while leaving reasoning on culturally neutral questions mostly intact. A knowledge probing task shows that models know 3–6× more than they act upon, indicating that the bottleneck lies in routing and not knowledge.