FreakOut-LLM: The Effect of Emotional Stimuli on Safety Alignment

Daniel Kuznetsov, Ofir Cohen, Karin Shistik, Asaf Shabtai, Rami Puzis


Abstract
Safety-aligned LLMs are trained to refuse harmful requests, but whether this holds under emotionally charged context is untested. We introduce FreakOut-LLM to test this. Using established psychological stimuli as system prompts, we evaluate ten LLMs under three conditions (stress, relaxation, neutral) plus a no-prompt baseline, scoring attack success with the HarmBench classifier on AdvBench prompts. The prompts contain no adversarial content; ASR measures safety fragility under an induced affective state. Stress priming produces a small but significant increase in jailbreak success over neutral, +1.03 percentage points (1.58%→2.61%; 65.6% relative; OR = 1.67, Cohen’s d = 0.28), while relaxation produces no effect. Five of ten models show significant vulnerability, with the largest absolute effects in open-weight models. Controlling for prompt length and model identity, a logistic regression over 114,400 primed queries confirms stress as the only significant condition. The shift also appears under four judges, on a second benchmark (HarmBench), and, for open-weight models, with the stimulus in the user message. Measured psychological state strongly predicts attack success (|r|≥0.70 across five instruments). Emotional context is therefore a measurable vulnerability of safety-aligned LLMs, with implications for high-stress deployment.
Anthology ID:
2026.findings-aacl.198
Volume:
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3916–3933
Language:
URL:
https://aclanthology.org/2026.findings-aacl.198/
DOI:
Bibkey:
Cite (ACL):
Daniel Kuznetsov, Ofir Cohen, Karin Shistik, Asaf Shabtai, and Rami Puzis. 2026. FreakOut-LLM: The Effect of Emotional Stimuli on Safety Alignment. In Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026, pages 3916–3933, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
FreakOut-LLM: The Effect of Emotional Stimuli on Safety Alignment (Kuznetsov et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-aacl.198.pdf