Qitao Tan
Author directory2026
Q-realign: Piggybacking Realignment on Quantization for Safe and Efficient LLM Deployment
Qitao Tan | Xiaoying Song | Ningxi Cheng | Ninghao Liu | Xiaoming Zhai | Lingzi Hong | Yanzhi Wang | Zhen Xiang | Geng Yuan
Findings of the Association for Computational Linguistics: EMNLP 2026
Qitao Tan | Xiaoying Song | Ningxi Cheng | Ninghao Liu | Xiaoming Zhai | Lingzi Hong | Yanzhi Wang | Zhen Xiang | Geng Yuan
Findings of the Association for Computational Linguistics: EMNLP 2026
Public large language models (LLMs) are typically safety-aligned during pretraining, but task-specific fine-tuning often erodes this alignment, introducing significant safety risks. Existing defenses are largely coupled with the fine-tuning process, incurring high computational overhead and complex workflows. To address these challenges, we propose Q-realign, a post-hoc defense based on quantization. Departing from the conventional view of quantization as a safety-eroding compromise, we reframe it as a dual-purpose mechanism for both compression and safety recovery. This allows Q-realign to decouple safety alignment from fine-tuning and naturally piggyback onto modern deployment pipelines. Experiments across multiple models and datasets demonstrate that our method substantially reduces unsafe behaviors while preserving task performance, significantly improving memory and GPU efficiency. Notably, our approach can recover the safety alignment of a fine-tuned 7B LLM on a single RTX 4090 within 40 minutes. Overall, our work provides a practical, turnkey solution for safety-aware deployment. Our code is released at https://anonymous.4open.science/r/Q-realign-copy-0D71.
End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost
Qitao Tan | Xiaoying Song | Ningxi Cheng | Guoming Li | Jun Liu | Lingzi Hong | Caiwen Ding | Jundong Li | Xiaoming Zhai | Shaoyi Huang | Jin Lu | Ninghao Liu | Wei Niu | Geng Yuan
Findings of the Association for Computational Linguistics: EMNLP 2026
Qitao Tan | Xiaoying Song | Ningxi Cheng | Guoming Li | Jun Liu | Lingzi Hong | Caiwen Ding | Jundong Li | Xiaoming Zhai | Shaoyi Huang | Jin Lu | Ninghao Liu | Wei Niu | Geng Yuan
Findings of the Association for Computational Linguistics: EMNLP 2026
Quantization reduces the deployment cost of large language models (LLMs). However, post-training quantization (PTQ) cannot fine-tune model parameters and often degrades accuracy in low-bit settings, while quantization-aware training (QAT) requires backpropagation and incurs high memory overhead. We propose ZeroQAT, a zeroth-order QAT framework supporting both weight and activation quantization. By using forward-only gradient estimation, ZeroQAT removes backpropagation and substantially reduces memory and computation while retaining end-to-end optimization benefits. We further introduce a lightweight variant of ZeroQAT for quantized fine-tuning, which freezes and pre-quantizes most parameters to further cut memory usage. Experiments show that ZeroQAT consistently outperforms representative PTQ and QAT baselines while requiring significantly less memory. For example, ZeroQAT enables fine-tuning of a 13B model at extremely low bit-widths (e.g., 2-4 bits) on a single 8GB GPU, and even allows fine-tuning a 6.7B model on a OnePlus 12 smartphone, demonstrating its practicality for end-to-end QAT on resource-limited edge devices.
Ask or Answer: A Decision Framework for Multi-Turn Health Misinformation Intervention
Xiaoying Song | Anirban Saha Anik | Jinyu Liu | Qitao Tan | Geng Yuan | Lingzi Hong
Findings of the Association for Computational Linguistics: EMNLP 2026
Xiaoying Song | Anirban Saha Anik | Jinyu Liu | Qitao Tan | Geng Yuan | Lingzi Hong
Findings of the Association for Computational Linguistics: EMNLP 2026
Correcting health misinformation in dialogue requires more than producing a factual rebuttal: users differ in what they know, what they believe, and what they need to hear, so an effective intervention often depends on first asking the right clarifying question. Yet existing methods either respond immediately or probe indiscriminately, treating clarification as either unnecessary or always beneficial. We propose Reward-Optimized Probe-and-Respond (RO-PnR), a framework that learns when asking is worth its cost. At each turn, RO-PnR chooses between probing for more information and committing to a final correction, guided by a turn-level reward that weighs the expected gain from probing against its interaction cost. To capture how user heterogeneity affects probing value, we model each simulated user with a latent state along health literacy and belief commitment. Experiments show that RO-PnR achieves the highest cost-adjusted utility across three health-misinformation datasets and three base models, using ∼30% fewer turns than always-probe baselines.