Ziyu Ge
Author directory2026
RabakBench: Evaluating Safety Guardrails in Singapore’s Multilingual Vernaculars
Gabriel Chua | Leanne Tan | Ziyu Ge | Roy Ka-Wei Lee
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Gabriel Chua | Leanne Tan | Ziyu Ge | Roy Ka-Wei Lee
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
LLM safety guardrails often behave unevenly in localized multilingual settings, especially with code-mixed vernaculars and non-Latin scripts. We introduce RabakBench, a Singapore-localized multilingual safety benchmark for Singlish, Chinese, Malay, and Tamil, built through a semi-automated, human-validated Generate–Label–Translate pipeline. The benchmark contains 1,341 unique examples in four language versions, yielding 5,364 entries across six harm categories, with LLM-assisted labels validated against human consensus. Evaluating 13 commercial and open-source guardrails reveals substantial cross-language and system-level variation, highlighting localized and script-sensitive safety gaps in the Singaporean context. RabakBench provides both a diagnostic resource and a reproducible, human-in-the-loop framework for constructing localized safety benchmarks.
2025
LionGuard 2: Building Lightweight, Data-Efficient & Localised Multilingual Content Moderators
Leanne Tan | Gabriel Chua | Ziyu Ge | Roy Ka-Wei Lee
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
Leanne Tan | Gabriel Chua | Ziyu Ge | Roy Ka-Wei Lee
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
Modern moderation systems increasingly support multiple languages, but often fail to address localisation and low-resource variants—creating safety gaps in real-world deployments. Small models offer a potential alternative to large LLMs, yet still demand considerable data and compute. We present LionGuard 2, a lightweight, multilingual moderation classifier tailored to the Singapore context, supporting English, Chinese, Malay, and partial Tamil. Built on pre-trained OpenAI embeddings and a multi-head ordinal classifier, LionGuard 2 outperforms several commercial and open-source systems across 17 benchmarks, including both Singapore-specific and public English datasets. The system is actively deployed within the Singapore Government, demonstrating practical efficacy at scale. Our findings show that high-quality local data and robust multilingual embeddings can achieve strong moderation performance, without fine-tuning large models. We release our model weights and part of our training data to support future work on LLM safety.