Bank statements are the primary evidentiary record in financial auditing, yet they reach auditors as heterogeneous PDFs—some digitally typeset, others scanned—transcribed by hand into spreadsheets and interrogated with brittle macros. We describe a system, built with and deployed for an industry partner, that replaces this workflow with (i) a multi-tier parser—a relational data conversion pipeline that routes digital pages to a deterministic table extractor (PyMuPDF) and scanned pages to an OCR/layout service (LlamaParse), standardizes column headers to a canonical schema, and repairs extraction errors with a balance-equation reconciliation engine; and (ii) a natural-language-to-SQL query layer over the resulting database, whose design choices we evaluate by varying context injection. On 23 real statements (20 banks, 1,672 transactions) the pipeline attains 0.9952 transaction-level accuracy. We compare five context-injection strategies across two model generations on 150 auditing queries with hand-authored gold SQL, scored by execution accuracy. Grounding the model on our parser’s structured context consistently helps, whereas raw-text context can degrade accuracy below the Zero-Context baseline; the benefit of self-correcting scaffolding and retrieval, while real, shrinks as the base model improves, so model choice often dominates agent complexity on simple schemas. We release code, two privacy-preserving sample statements, and a sample query set.

In this work, we introduce BanglaBERT, a BERT-based Natural Language Understanding (NLU) model pretrained in Bangla, a widely spoken yet low-resource language in the NLP literature. To pretrain BanglaBERT, we collect 27.5 GB of Bangla pretraining data (dubbed ‘Bangla2B+’) by crawling 110 popular Bangla sites. We introduce two downstream task datasets on natural language inference and question answering and benchmark on four diverse NLU tasks covering text classification, sequence labeling, and span prediction. In the process, we bring them under the first-ever Bangla Language Understanding Benchmark (BLUB). BanglaBERT achieves state-of-the-art results outperforming multilingual and monolingual models. We are making the models, datasets, and a leaderboard publicly available at https://github.com/csebuetnlp/banglabert to advance Bangla NLP.