Fact-Checking the Jailbreak: A Claim-Level Framework for Structured Misinformation Assessment

Jia Yee Tan, Rong Fu, Yongtai Liu, Kaiyi Hu, Simon James Fong


Abstract
Large language models (LLMs) remain vulnerable to jailbreak attacks that elicit structured, credible misinformation. Existing benchmarks primarily track binary refusals, often overlooking the severity and exposure risk of the misinformation payload itself. In this work, we introduce Misinfo-Jailbreak Bench, a comprehensive framework that systematically maps diverse attack strategies to a claim-level scoring pipeline for precise risk quantification. Building on this diagnostic foundation, we propose Selective LoRA Ethical Surgery, a parameter-efficient defense mechanism. Instead of broad parameter updates, our approach identifies safety-relevant subspaces via contrastive representations to confine LoRA edits and employs inference-time gating to preserve benign capabilities. Experimental results on state-of-the-art models (e.g., Qwen2.5, Llama-4) demonstrate that our method reduces high-severity risk by 60-70% while limiting general capability degradation to less than 1%, offering a precise and minimally invasive paradigm for post-hoc alignment.
Anthology ID:
2026.findings-emnlp.23
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
429–448
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.23/
DOI:
Bibkey:
Cite (ACL):
Jia Yee Tan, Rong Fu, Yongtai Liu, Kaiyi Hu, and Simon James Fong. 2026. Fact-Checking the Jailbreak: A Claim-Level Framework for Structured Misinformation Assessment. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 429–448, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Fact-Checking the Jailbreak: A Claim-Level Framework for Structured Misinformation Assessment (Tan et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.23.pdf