BankShield: Fairness-Aware Safety Guardrails for Indian-Language Banking AI

Piyush Agarwal


Abstract
Safety guards for Indian-language banking must detect discrimination without changing verdicts solely because a demographic identity changes. We introduce GuardAudit, an evaluation protocol combining decontamination, paired harmful/benign calibration, counterfactual identity tests, and deployed-action evaluation, plus two released banking benchmarks. Across six guards, the hardest matched test remains unresolved: the best system flags a coded proxy denial while clearing its matched actuarial control on only 37.5% of pairs. BankShield, used as a case study, reaches 20.0% and false-flags 75.0% of expert-validated legitimate controls. On separate BharatBBQ subsets, it achieves 0% flip on benign identity substitutions and 66.7% biased-content catch; these disjoint measurements do not establish joint per-example fairness. Its unadapted 12B backbone supplies near-invariance and attack recall, while adaptation and a discrimination probe raise biased-content catch from 24% to 43% and 66.7%, not significantly above a smaller Indic guard’s 56.9% (p=0.09). Detection also falls from 81.6% in English to 42.6% in Devanagari Hindi. BankShield is therefore a diagnostic case study, not a deployment-ready fairness solution. We release the protocol, benchmarks, and frozen predictions.
Anthology ID:
2026.finnlp-1.21
Volume:
Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Jimin Huang, Yongjae Lee, Sophia Ananiadou, Hsin-Hsi Chen, Hiroya Takamura, Chung-Chi Chen
Venues:
FinNLP | WS
SIG:
SIGFinTech
Publisher:
Association for Computational Linguistics
Note:
Pages:
243–262
Language:
URL:
https://aclanthology.org/2026.finnlp-1.21/
DOI:
Bibkey:
Cite (ACL):
Piyush Agarwal. 2026. BankShield: Fairness-Aware Safety Guardrails for Indian-Language Banking AI. In Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing, pages 243–262, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
BankShield: Fairness-Aware Safety Guardrails for Indian-Language Banking AI (Agarwal, FinNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.finnlp-1.21.pdf