Oussama Chahed

Author directory

2026

Large language models (LLMs) analyze corporate financial statements fluently, yet routinely emit wrong figures while preserving a confident, well-formatted narrative—the false-precision problem. Prior benchmarks measure answer accuracy or attach claims to source cells, but do not explain why a financial number is wrong. We introduce a deterministic, XBRL-grounded oracle that recomputes the intermediate-management-balances (SIG) and ratio cascade from SEC EDGAR facts, and use it to score numeric fidelity per financial concept, with fact-level attribution and error propagation along the ratio dependency DAG. A pinned/unpinned prompt contrast, an any-standard-definition oracle for convention-dependent composites, and a rule-based error-mode classifier jointly separate definitional divergence (the model applies a different but sometimes valid convention) from unexplained residual numerical error. Across six open and proprietary models on 729 firm-years (and a 345 firm-year subset common to all six), we find: (i) simple two-fact ratios are computed faithfully by all but the weakest model, while multi-input composites separate the panel by capability—the residual hardest concept (ROCE) is executed reliably only by the strongest models; (ii) supplying the exact formula and source tags (pinning) helps every capable model and only hurts the weakest, inverting an apparent "computational limit" that turns out to be a weak-model artifact; and (iii) source-fact attribution and fidelity are distinct and can decouple—one model computes correctly yet cites its source facts almost never ("right answer, no receipts"), while the strongest models pair high fidelity with high attribution. Program-of-thought prompting does not change fidelity, and most wrong composites are local errors rather than propagated ones. Code, frozen model responses and full provenance are released under a DOI.
Search
Co-authors
    Venues
    Fix author