CORAL: Closed-loop Oracle Alignment for Agentic Tax Engine Modernization

Maarten De Raedt, Ahmed Abdullah, Abhinav Agrawal


Abstract
Modernizing legacy software is often framed as code migration, an approach that assumes access to source code and an understanding of system internals. In practice these assumptions fail: source may be inaccessible, data in proprietary formats, and documentation dispersed. We study an underexplored regime in which a replacement is aligned to a legacy system treated as a black-box oracle, using only its input–output behavior.

We present CORAL (Closed-loop Oracle Alignment), an agentic framework that aligns a modern U.S. individual income tax engine (Form 1040) to a production oracle (a system built over decades) by iteratively refining LLM-bootstrapped form schemas. Each round runs a scenario on both systems, compares e-file output field-by-field, and diagnoses discrepancies against an authoritative IRS knowledge base (KB), grounding every fix in published regulation rather than relying on model knowledge. An edit is kept only if re-execution shows no meaningful field-level F1 regression and all tests pass, a deterministic check rather than model self-assessment. Round-robin sequencing over under-covered forms then reaches the long tail of forms, not just the high-frequency ones a taxpayer shares with everyone.

Across 119 tax-expert-authored scenarios, each a complete taxpayer return, CORAL repairs defects from single-field corrections to reviving entire non-functional forms (53 fixed; 13 escalated to human review, ensuring ambiguous rules and input-data gaps are audited manually), lifting macro-F1 on a 30-scenario held-out test split, averaged over returns, from an LLM-bootstrapped start (0.17) to 0.90.
Anthology ID:
2026.finnlp-1.15
Volume:
Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Jimin Huang, Yongjae Lee, Sophia Ananiadou, Hsin-Hsi Chen, Hiroya Takamura, Chung-Chi Chen
Venues:
FinNLP | WS
SIG:
SIGFinTech
Publisher:
Association for Computational Linguistics
Note:
Pages:
180–192
Language:
URL:
https://aclanthology.org/2026.finnlp-1.15/
DOI:
Bibkey:
Cite (ACL):
Maarten De Raedt, Ahmed Abdullah, and Abhinav Agrawal. 2026. CORAL: Closed-loop Oracle Alignment for Agentic Tax Engine Modernization. In Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing, pages 180–192, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
CORAL: Closed-loop Oracle Alignment for Agentic Tax Engine Modernization (De Raedt et al., FinNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.finnlp-1.15.pdf