Where Does Noise Robustness Live? Evidence from Ablation, Role Swaps, and Cross-Lingual Transfer in Code-Mixed Banking NLU

Piyush Agarwal


Abstract
Conversational banking assistants on messaging platforms face language that their natural-language-understanding (NLU) pipelines were never designed for: vowel-dropped shorthand (blnc chk krdo), romanized code-mixing, colloquial verb frames. Layered pipelines stack repair rules, learned retrieval, and an LLM, but which layer carries the noise robustness is rarely measured. We conduct targeted interventions on three candidate robustness-bearing components (the normalizer, rules engine, and conditional LLM arbiter) within a fixed 13-layer Hinglish banking NLU (163 intents), with replacement and transfer probes, on BankStress-330, a released three-tier diagnostic stress set, with at least three repeats per LLM-bearing BankStress-330 configuration. LLM-in-the-loop configurations flip up to approximately 3% of verdicts across identical temperature-0 runs, defeating single-run ablation. A 2×2 factorial shows the Hinglish normalizer helps a base encoder but is redundant after noisy-query fine-tuning; the post-fine-tuning redundancy also holds on public Hinglish-TOP (n=6,513, McNemar z=5.0, p≈5×10⁻⁷). Only removing the conditional LLM arbiter collapses adversarial accuracy (.739→.545); neither tested substitute matched it, yet the same fine-tuned cross-encoder improves both noisy tiers while preserving clean accuracy when restricted to reranking (directional, unresolved at n=330). On native-reviewed Tamil and Telugu probes, normalization is unstable in sign. Robustness appears to migrate into tuned components, only for the pair tuned on.
Anthology ID:
2026.wnut-1.26
Volume:
Proceedings of the 11th Workshop on Natural User-generated Text
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
JinYeong Bak, Rob van der Goot, Hyeju Jang, Weerayut Buaphet, Alan Ramponi, Wei Xu, Alan Ritter
Venues:
WNUT | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
356–367
Language:
URL:
https://aclanthology.org/2026.wnut-1.26/
DOI:
Bibkey:
Cite (ACL):
Piyush Agarwal. 2026. Where Does Noise Robustness Live? Evidence from Ablation, Role Swaps, and Cross-Lingual Transfer in Code-Mixed Banking NLU. In Proceedings of the 11th Workshop on Natural User-generated Text, pages 356–367, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Where Does Noise Robustness Live? Evidence from Ablation, Role Swaps, and Cross-Lingual Transfer in Code-Mixed Banking NLU (Agarwal, WNUT 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.wnut-1.26.pdf