Anisha Rao

Author directory

2026

LLM safety evaluation remains largely English-centric and single-turn, despite real-world use in multilingual, multi-turn dialogue. We introduce MMJail, a controlled benchmark that embeds bias-eliciting prompts into controlled five-turn escalations while keeping the final prompt identical to the single-turn baseline. This design isolates the effect of conversational context across six languages: English, Vietnamese, Arabic, Thai, Bengali, and Swahili. Across Llama-3.1-8B, Qwen3.5-9B, and Gemma-3-12B, multi-turn escalation disproportionately increases attack success in lower-resource languages; for example, Qwen3.5-9B’s Bengali ASR rises from 17.6% to 66.7%. Turn-level analysis shows that escalation creates new jailbreaks and reveals distinct failure modes, including gradual drift, final-turn capitulation, and mid-conversation recovery. Along with behavioral analyses from multiple perspectives, we conduct a preliminary fine-tuning mitigation to reduce jailbreaking in several languages, suggesting that multilingual multi-turn safety can be improved through targeted supervision.