DealTrace: Localizing and Intervening on Failures in Agentic Financial Reasoning

Alina Hyk


Abstract
Analyzing a financial deal is a long, multi-step task: an analyst reads a narrative memorandum against the Excel model behind it, rebuilds the forecast under the valuation, and writes a recommendation. LLM agents increasingly perform such work, but financial benchmarks measure isolated skills or final answers and do not show where a long analysis fails. We present DealTrace, a benchmark built on real deal packages, each pairing a confidential information memorandum with its financial model. A five-stage pipeline—extract, reconcile, project a masked forecast, assess market context, recommend—follows the analyst workflow; every clause of the recommendation cites the stage that produced it, and knockout and repair interventions attribute the outcome to individual stages. Across six frontier models, three scaffolding conditions, and four model judges, extraction, reconciliation, and grounding are reliable, and forecasting is the decisive stage: it is the strongest correlate of judged quality, the costliest stage to remove, and the only repair that improves every model, after which the models reach nearly the same score. Models flag discrepancies accurately but rarely use them in the verdict, a failure that outcome-level evaluation cannot detect. Scaffolding improves stage-level performance far more than the final score, and a critique pass adds cost without reliable gain. DealTrace evaluates the analysis process itself, not only its final answer.
Anthology ID:
2026.finnlp-1.52
Volume:
Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Jimin Huang, Yongjae Lee, Sophia Ananiadou, Hsin-Hsi Chen, Hiroya Takamura, Chung-Chi Chen
Venues:
FinNLP | WS
SIG:
SIGFinTech
Publisher:
Association for Computational Linguistics
Note:
Pages:
611–631
Language:
URL:
https://aclanthology.org/2026.finnlp-1.52/
DOI:
Bibkey:
Cite (ACL):
Alina Hyk. 2026. DealTrace: Localizing and Intervening on Failures in Agentic Financial Reasoning. In Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing, pages 611–631, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
DealTrace: Localizing and Intervening on Failures in Agentic Financial Reasoning (Hyk, FinNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.finnlp-1.52.pdf