Guangquan Zhang
Author directory2026
Rethinking Cross-Modality Prompt Transfer: Geometric Priors over Linguistic Semantics
Ningyuan Zhang | Jie Lu | Keqiuyin Li | Guangquan Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Ningyuan Zhang | Jie Lu | Keqiuyin Li | Guangquan Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Cross-Modality Prompt Transfer (CMPT) has been explored to address the initialization instability and data scarcity challenges inherent in prompt tuning. Its empirical success is typically interpreted as the transfer of linguistic semantics acquired during language-based prompt pretraining. In this paper, we revisit this interpretation and explore what actually transfers when language-pretrained prompts are adapted to the visual modality. Through controlled empirical analyses, we show that transfer performance does not consistently correlate with source-task linguistic performance. Instead, transferability is better explained by the geometric priors of prompt vectors, particularly stable rank and spatial direction relative to target features. To further validate this interpretation, we directly construct prompts with favorable geometry without relying on language pretraining. The resulting prompts achieve competitive transfer performance against language-pretrained prompts across diverse visual tasks, supporting prompt geometry as a stronger empirical explanation of transferability.
TPA: Next Token Probability Attribution for Detecting Hallucinations in RAG
Pengqian Lu | Jie Lu | Anjin Liu | Guangquan Zhang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Pengqian Lu | Jie Lu | Anjin Liu | Guangquan Zhang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Detecting hallucinations in Retrieval-Augmented Generation remains a challenge. Prior approaches attribute hallucinations to a binary conflict between internal knowledge stored in FFNs and the retrieved context. However, this perspective is incomplete, failing to account for the impact of other components of the LLM, such as the user query, previously generated tokens, the self token, and the final LayerNorm adjustment. To comprehensively capture the impact of these components on hallucination detection, we propose TPA which mathematically attributes each token’s probability to seven distinct sources: Query, RAG Context, Past Token, Self Token, FFN, Final LayerNorm, and Initial Embedding. This attribution quantifies how each source contributes to the generation of the next token. Specifically, we aggregate these attribution scores by Part-of-Speech (POS) tags to quantify the contribution of each model component to the generation of specific linguistic categories within a response. By leveraging these patterns, such as detecting anomalies where Nouns rely heavily on LayerNorm, TPA effectively identifies hallucinated responses. Extensive experiments show that TPA achieves state-of-the-art performance.