Fayz Siddiqui
Author directory2026
Deploying LLM Agents for Auditable Complaint Investigation in Banking
John Hartley | Conor Brian Hamill | Azam Din | Fayz Siddiqui | Kingsley Nketia Acheampong | Greig A Cowan | Raad Khraishi
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
John Hartley | Conor Brian Hamill | Azam Din | Fayz Siddiqui | Kingsley Nketia Acheampong | Greig A Cowan | Raad Khraishi
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Complaint investigation in retail banking requires handlers to synthesise evidence from heterogeneous enterprise systems into policy-aligned summaries. We present a production-deployed tool-augmented LLM agent at a major UK retail bank that assists this process under human review. Given a complaint record, the agent queries 16 typed, schema-validated tools and returns an investigation summary with a source-cited chronological evidence timeline. We evaluate the system using SME assessment, LLM-as-a-Judge monitoring, rework measurement, and downstream impact testing. On 187 paired complaints, SMEs judged the agent to match or exceed historical human-written production investigations in 90% of cases across faithfulness, relevance, completeness, and coherence. Median investigation time falls from 30 minutes to 23 seconds. Operational controls include daily automated monitoring, pre-defined thresholds, hallucination flagging, and periodic human audit. These results demonstrate that constrained single-agent LLMs with typed tools are viable for scalable, auditable complaint investigation in regulated environments.