2026

Analyzing a financial deal is a long, multi-step task: an analyst reads a narrative memorandum against the Excel model behind it, rebuilds the forecast under the valuation, and writes a recommendation. LLM agents increasingly perform such work, but financial benchmarks measure isolated skills or final answers and do not show where a long analysis fails. We present DealTrace, a benchmark built on real deal packages, each pairing a confidential information memorandum with its financial model. A five-stage pipeline—extract, reconcile, project a masked forecast, assess market context, recommend—follows the analyst workflow; every clause of the recommendation cites the stage that produced it, and knockout and repair interventions attribute the outcome to individual stages. Across six frontier models, three scaffolding conditions, and four model judges, extraction, reconciliation, and grounding are reliable, and forecasting is the decisive stage: it is the strongest correlate of judged quality, the costliest stage to remove, and the only repair that improves every model, after which the models reach nearly the same score. Models flag discrepancies accurately but rarely use them in the verdict, a failure that outcome-level evaluation cannot detect. Scaffolding improves stage-level performance far more than the final score, and a critique pass adds cost without reliable gain. DealTrace evaluates the analysis process itself, not only its final answer.
Search
Co-authors
    Venues
    Fix author