Zhenghua Wang
2025
Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective
Tianlong Li
|
Zhenghua Wang
|
Wenhao Liu
|
Muling Wu
|
Shihan Dou
|
Changze Lv
|
Xiaohua Wang
|
Xiaoqing Zheng
|
Xuanjing Huang
Proceedings of the 31st International Conference on Computational Linguistics
The recent surge in jailbreaking attacks has revealed significant vulnerabilities in Large Language Models (LLMs) when exposed to malicious inputs. While various defense strategies have been proposed to mitigate these threats, there has been limited research into the underlying mechanisms that make LLMs vulnerable to such attacks. In this study, we suggest that the self-safeguarding capability of LLMs is linked to specific activity patterns within their representation space. Although these patterns have little impact on the semantic content of the generated text, they play a crucial role in shaping LLM behavior under jailbreaking attacks. Our findings demonstrate that these patterns can be detected with just a few pairs of contrastive queries. Extensive experimentation shows that the robustness of LLMs against jailbreaking can be manipulated by weakening or strengthening these patterns. Further visual analysis provides additional evidence for our conclusions, providing new insights into the jailbreaking phenomenon. These findings highlight the importance of addressing the potential misuse of open-source LLMs within the community.
2024
Searching for Best Practices in Retrieval-Augmented Generation
Xiaohua Wang
|
Zhenghua Wang
|
Xuan Gao
|
Feiran Zhang
|
Yixin Wu
|
Zhibo Xu
|
Tianyuan Shi
|
Zhengyuan Wang
|
Shizheng Li
|
Qi Qian
|
Ruicheng Yin
|
Changze Lv
|
Xiaoqing Zheng
|
Xuanjing Huang
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Retrieval-augmented generation (RAG) techniques have proven to be effective in integrating up-to-date information, mitigating hallucinations, and enhancing response quality, particularly in specialized domains. While many RAG approaches have been proposed to enhance large language models through query-dependent retrievals, these approaches still suffer from their complex implementation and prolonged response times. Typically, a RAG workflow involves multiple processing steps, each of which can be executed in various ways. Here, we investigate existing RAG approaches and their potential combinations to identify optimal RAG practices. Through extensive experiments, we suggest several strategies for deploying RAG that balance both performance and efficiency. Moreover, we demonstrate that multimodal retrieval techniques can significantly enhance question-answering capabilities about visual inputs and accelerate the generation of multimodal content using a “retrieval as generation” strategy.
Search
Fix data
Co-authors
- Xuan-Jing Huang (黄萱菁) 2
- Changze Lv 2
- Xiaohua Wang 2
- Xiaoqing Zheng 2
- Shihan Dou 1
- show all...