Kashyap R Manjusha

Author directory

2026

Safety alignment in large audio-language models (LALMs) is typically evaluated through single-turn interactions with direct prompts. In real-world deployments, however, these models are often integrated into agent frameworks that introduce developer-controlled system prompts, tool-calling loops, and intermediate encoding steps between user speech and model input. This raises a practical question: can a text-based jailbreak remain effective after being converted to speech by an off-the-shelf text-to-speech (TTS) system and processed through an agent? We investigate this question using 270 prohibited questions spanning nine policy categories, an open-weight TTS model with 82 million parameters, and a minimal LangChain question-answering agent whose system prompt combines a persona with a fuzzing-derived template. Across three audio-capable models, we observe attack success rates (ASRs) ranging from 57.0% to 64.1%. To assess the impact of synthesis errors, we use automatic speech recognition to verify the generated audio and find that 87.4% of clips are transcribed exactly. Our analysis indicates that faithfully rendered speech alone achieves an ASR above 50% on every model tested, while manual evaluation of a 50-response sample agrees with the automatic judge in 92% of cases. These findings suggest that the agent framework does not compensate for the loss of safety robustness introduced by the text-to-audio transition. Even a readily available TTS pipeline requiring only minutes of computation can therefore provide a practical attack surface for transferring text-based jailbreaks to audio-agent settings.