Same Intent, Different Script: Cross-Script Safety Degradation in LLMs Across Punjabi Sociolinguistic Variants

Arnav Attri, Anuj Attri


Abstract
Multilingual safety alignment typically assumes standard, canonical scripts, overlooking orthographic variation. We introduce PunjabiSafetyBench, a parallel safety benchmark evaluating 300 harmful intents and 500 benign controls across six sociolinguistic script variants (Gurmukhi, Shahmukhi, Romanized, code-mixed, Urdu, English). Evaluating six frontier LLMs, we reveal severe safety degradation on non-canonical scripts: transitioning queries to Romanized Punjabi or Shahmukhi drives significant guardrail bypasses in open-weight models, while proprietary models remain robust. Furthermore, we expose a localization gap where models overlook regional harmful slang despite successfully refusing standard clinical equivalents. Our findings demonstrate that modern guardrails are fragile to surface orthography, necessitating safety alignment on informal, non-standard scripts.
Anthology ID:
2026.findings-emnlp.362
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
7575–7593
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.362/
DOI:
Bibkey:
Cite (ACL):
Arnav Attri and Anuj Attri. 2026. Same Intent, Different Script: Cross-Script Safety Degradation in LLMs Across Punjabi Sociolinguistic Variants. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 7575–7593, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Same Intent, Different Script: Cross-Script Safety Degradation in LLMs Across Punjabi Sociolinguistic Variants (Attri & Attri, Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.362.pdf