Luis Belmar Letelier
Author directory2026
ForensicBench: Evaluating Agentic LLMs on Journal-Entry Fraud Detection
Guy Stéphane Waffo Dzuyo | Gaël Guibon | Christophe Cerisara | Luis Belmar Letelier
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Guy Stéphane Waffo Dzuyo | Gaël Guibon | Christophe Cerisara | Luis Belmar Letelier
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Accounting fraud costs organizations billions annually, yet auditors rarely confront it as a single suspicious posting. Instead, they investigate coordinated *fraud schemes* such as fictitious vendor payments, ghost-employee payroll, and inflated revenues that span many journal entries, each individually routine, and only become visible when understood against the underlying accounting process. Existing AI systems either score isolated entries as anomalies or mine company-level disclosures; none evaluate whether a large language model (LLM) agent can query a live ledger, recover coordinated fraud patterns, and assign the correct scheme type. We release **ForensicBench**: the first benchmark for evaluating agentic LLMs on *scheme-level* journal-entry fraud detection. The dataset, **Forensic Ledger**, extends DataSynth with five injected fraud scheme types and scheme-level labels linking journal entries to coordinated instances. We evaluate twelve open-weight models under a single reference agent scaffold on a public leaderboard: among them, the best reaches only 34.7% Entry-F1, and Type-F1 is up to 13 points lower. We publicly release the dataset, evaluation harness, and a reference agentic baseline