A Strong Harness Narrows the Model Gap: A Comparative Evaluation for Legislative Document-to-XML Conversion

Palvika Bansal, Rittika Jindal, Ruichong Wang


Abstract
Legal-content reproduction requires matching the source exactly in section hierarchy, verbatim text, citations, and amendment spans; a dropped citation or mislabeled amendment carries into the products lawyers rely on. We report an industrial pipeline converting U.S. state legislation into strict, schemaconstrained XML, and hypothesize that its accuracy comes from harness engineering, the deterministic tooling built around a model, more than the model itself. Our controlled ablation crosses three harnesses (zero-shot, a single tool-using agent, and a decomposed multiagent pipeline) with three Claude models on 153 gold bills from two jurisdictions. The harness narrows the model gap: the weakest model in the full pipeline beats the strongest model alone (0.948 versus 0.806, paired bootstrap p less than 0.001). Because the two agentic harnesses share identical tools, their gap comes from orchestration: decomposition plus the persisted state and per-agent context it brings. Across four scored dimensions, the model spread at zero-shot (0.68 to 0.81) narrows to 0.95 at the full pipeline; the residual sits almost entirely in citation normalization, where the canonical identifier is retrieved from an external authority rather than generated from the weights, so a deterministic lookup tool supplies the exactness that a larger model does not.
Anthology ID:
2026.nllp-1.17
Volume:
Proceedings of the Natural Legal Language Processing Workshop 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Nikolaos Aletras, Ilias Chalkidis, Leslie Barrett, Cătălina Goanță, Daniel Preoțiuc-Pietro, Gerasimos Spanakis, Shanshan Xu
Venues:
NLLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
183–198
Language:
URL:
https://aclanthology.org/2026.nllp-1.17/
DOI:
Bibkey:
Cite (ACL):
Palvika Bansal, Rittika Jindal, and Ruichong Wang. 2026. A Strong Harness Narrows the Model Gap: A Comparative Evaluation for Legislative Document-to-XML Conversion. In Proceedings of the Natural Legal Language Processing Workshop 2026, pages 183–198, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
A Strong Harness Narrows the Model Gap: A Comparative Evaluation for Legislative Document-to-XML Conversion (Bansal et al., NLLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.nllp-1.17.pdf