Zixuan Xu
Author directory2026
SaFeR-ToolKit: Structured Reasoning via Virtual Tool Calling for Multimodal Safety
Zixuan Xu | Tiancheng He | Huahui Yi | Kun Wang | Xi Chen | Gongli Xi | Qiankun Li | Kang Li | Yang Liu | Zhigang Zeng
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Zixuan Xu | Tiancheng He | Huahui Yi | Kun Wang | Xi Chen | Gongli Xi | Qiankun Li | Kang Li | Yang Liu | Zhigang Zeng
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Vision–language models remain susceptible to multimodal jailbreaks and over-refusal because safety hinges on both visual evidence and user intent, while many alignment pipelines supervise only the final response. To address this, we present SaFeR-ToolKit, which formalizes safety decision-making as an inspectable protocol: during offline data synthesis, a planner specifies a persona, a **Perception**→**Reasoning**→**Decision** tool set, and a constrained transition graph, while a responder emits a typed key–value tool trace before the final answer. To learn this protocol, we train a single policy with a three-stage curriculum (SFT→DPO→GRPO), where GRPO directly supervises tool usage beyond answer-level feedback. We contribute (**I**) the first tool-based safety reasoning dataset of 30,654 training entries (SFT 6k, DPO 18.6k, GRPO 6k) plus 1k held-out evaluation, and (**II**) experiments on Qwen2.5-VL showing large gains in Safety/Helpfulness/Reasoning Rigor on 3B (29.39/45.04/4.98 → 84.40/71.13/78.87) and 7B (53.21/52.92/19.26 → 86.34/80.79/85.34), while preserving general capabilities (3B: 58.67 → 59.21; 7B: 66.39 → 66.81). Code: [https://github.com/Duebassx/SafeR_Toolkit](https://github.com/Duebassx/SafeR_Toolkit).