Regression-Aware Statistically Gated Policy Updating for Korean Financial LLM Input Guardrails

Eunjung Cho, Dae-Young Park, Minwoo Kim


Abstract
Financial institutions must prevent personally identifiable information (PII) and adversarial instructions from entering large language models (LLMs). We present regression-aware policy maintenance for Korean financial LLM input guardrails, supported by a cost-ordered Regex–NER–LLM cascade. On a privacy-preserving controlled set of 1,000 synthetic attacks and 200 routine requests, four manual iterations reduce in-distribution attack success rate (ASR) from 8.9% to 0.3% with a 0.5% false-positive rate (FPR) on the original tuning set. On a held-out split, a policy tuned only on the tuning split achieves 0/300 attack successes and 1/60 false positives, with Wilson 95% CIs of [0, 1.26]% and [0.29, 8.86]%, respectively. For policy maintenance, an LLM generates candidate patches, while paired McNemar tests, an FPR constraint, and cumulative regression checks determine acceptance or rejection. Paired regression testing detects an intermediate regression introduced in manual Round 3, while the gate rejects all five automatically generated candidates, preventing statistically unsupported changes from entering the policy. For boundary diagnosis, the rejected fifth-round candidate yields a pre-mitigation overall FPR of 28.9% across 350 benign examples, including 150 hard negatives, revealing context-blind Regex blocking and LLM-judge over-blocking hidden by routine benign requests. These results support regression-aware policy maintenance in the controlled setting while highlighting the need for broader boundary and deployment-specific evaluation.
Anthology ID:
2026.finnlp-1.36
Volume:
Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Jimin Huang, Yongjae Lee, Sophia Ananiadou, Hsin-Hsi Chen, Hiroya Takamura, Chung-Chi Chen
Venues:
FinNLP | WS
SIG:
SIGFinTech
Publisher:
Association for Computational Linguistics
Note:
Pages:
434–440
Language:
URL:
https://aclanthology.org/2026.finnlp-1.36/
DOI:
Bibkey:
Cite (ACL):
Eunjung Cho, Dae-Young Park, and Minwoo Kim. 2026. Regression-Aware Statistically Gated Policy Updating for Korean Financial LLM Input Guardrails. In Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing, pages 434–440, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Regression-Aware Statistically Gated Policy Updating for Korean Financial LLM Input Guardrails (Cho et al., FinNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.finnlp-1.36.pdf