Liang Lin
Author directoryOther people with similar names: Liang Lin, Liang Lin
Unverified author pages with similar names: Liang Lin
2026
HearSay Benchmark: Do Audio LLMs Leak What They Hear?
Jin Wang | Kaiwen Luo | Liang Lin | Weiliu Wang | Yitian Chen | Moayad Aloqaily | Xuehai Tang | Zhenhong Zhou | Kun Wang | Li Sun | Qingsong Wen
Findings of the Association for Computational Linguistics: ACL 2026
Jin Wang | Kaiwen Luo | Liang Lin | Weiliu Wang | Yitian Chen | Moayad Aloqaily | Xuehai Tang | Zhenhong Zhou | Kun Wang | Li Sun | Qingsong Wen
Findings of the Association for Computational Linguistics: ACL 2026
While Audio Large Language Models (ALLMs) have achieved remarkable progress in understanding and generation, their potential privacy implications remain largely unexplored. This paper takes the first step to investigate whether ALLMs inadvertently leak user privacy solely through acoustic voiceprints and introduces HearSay, a comprehensive benchmark constructed from over 22,000 real-world audio clips. To ensure data quality, the benchmark is meticulously curated through a rigorous pipeline involving automated profiling and human verification, guaranteeing that all privacy labels are grounded in factual records. Extensive experiments on HearSay yield three critical findings:Significant Privacy Leakage: ALLMs inherently extract private attributes from voiceprints, reaching 92.89% accuracy on gender and effectively profiling social attributes.Insufficient Safety Mechanisms: Alarmingly, existing safeguards are severely inadequate; most models fail to refuse privacy-intruding requests, exhibiting near-zero refusal rates for physiological traits.Reasoning Amplifies Risk: Chain-of-Thought (CoT) reasoning exacerbates privacy risks in capable models by uncovering deeper acoustic correlations.These findings expose critical vulnerabilities in ALLMs, underscoring the urgent need for targeted privacy alignment.The codes and dataset are available at https://github.com/JinWang79/HearSay_Benchmark
RSA-Bench: Benchmarking Audio Large Models in Real-World Acoustic Scenarios
Yibo Zhang | Kaiwen Luo | Liang Lin | Shilinlu Yan | Jin Wang | Yaoqi Guo | Yitian Chen | Yalan Qin | Zhenhong Zhou | Kun Wang | Li Sun
Findings of the Association for Computational Linguistics: ACL 2026
Yibo Zhang | Kaiwen Luo | Liang Lin | Shilinlu Yan | Jin Wang | Yaoqi Guo | Yitian Chen | Yalan Qin | Zhenhong Zhou | Kun Wang | Li Sun
Findings of the Association for Computational Linguistics: ACL 2026
While Audio Large Models (ALLMs) have achieved remarkable proficiency, their robustness remains brittle in real-world deployment. Existing evaluations largely rely on synthetic Gaussian noise or simplistic single-source interference, failing to capture the intricate, multi-layered acoustic dynamics—or “Acoustic Ecology”—that characterize authentic physical environments. To bridge this ecological gap, we introduce RSA-Bench, a comprehensive robustness benchmark designed to stress-test ALLMs through high-fidelity auditory scene simulations. Unlike traditional methods, we construct evaluation samples by naturally superimposing diverse environmental soundscapes—spanning Pasture, Extreme Weather, Classroom, and Outdoors—onto clean speech signals across a spectrum of interference intensities. By evaluating models on six core tasks ranging from fundamental perception to complex reasoning, our study unveils three macro-level insights: (I) The Perception-Cognition Gap: Models maintain relative resilience in low-level recognition but suffer a functional collapse in high-order reasoning tasks under stress; (II) Scenario Sensitivity: “Vocal-like” interference (e.g., children playing) proves significantly more destructive than mechanical noise, challenging the model’s auditory attention mechanisms; and (III) The Denoising Paradox: Standard speech enhancement often exacerbates performance degradation, as ALLMs prove highly sensitive to the semantic distortions introduced by denoising artifacts.
Backdoor Collapse: Eliminating Unknown Threats Via Known Backdoor Aggregation In Language Models
Liang Lin | Miao Yu | Moayad Aloqaily | Zhenhong Zhou | Kun Wang | Linsey Pang | Prakhar Mehrotra | Qingsong Wen
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Liang Lin | Miao Yu | Moayad Aloqaily | Zhenhong Zhou | Kun Wang | Linsey Pang | Prakhar Mehrotra | Qingsong Wen
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Backdoor attacks are a significant threat to large language models (LLMs), often embedded via public checkpoints, yet existing defenses rely on impractical assumptions about trigger settings. To address this challenge, we propose Locphylax, a defense framework that requires no prior knowledge of trigger settings. Locphylax is based on the key observation that when deliberately injecting known backdoors into an already-compromised model, both existing unknown and newly injected backdoors aggregate in the representation space. Locphylax leverages this through a two-stage process: first, aggregating backdoor representations by injecting known triggers, and then, performing recovery fine-tuning to restore benign outputs. Extensive experiments across multiple LLM architectures demonstrate that: (I) Locphylax reduces the average Attack Success Rate to 4.41% across multiple benchmarks, outperforming existing baselines by 28.1%–69.3%. (II) Clean accuracy and utility are preserved within 0.5% of the original model, ensuring negligible impact on legitimate tasks. (III) The defense generalizes across different types of backdoors, confirming its robustness in practical deployment scenarios. Our code is available at https://anonymous.4open.science/r/Locphylax.
SEE: Signal Embedding Energy for Quantifying Noise Interference in Large Audio Language Models
Yuanhe Zhang | Jiayu Tian | Yibo Zhang | Shilinlu Yan | Liang Lin | Zhenhong Zhou | Li Sun | Sen Su
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yuanhe Zhang | Jiayu Tian | Yibo Zhang | Shilinlu Yan | Liang Lin | Zhenhong Zhou | Li Sun | Sen Su
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Large Audio Language Models (LALMs) have been widely applied in real-time scenarios, such as in-car assistants and online meeting comprehension. In practice, audio inputs are often corrupted by device and environmental noise, leading to performance degradation. However, existing LALM studies on noise lack quantitative analysis and rely mainly on intuition and empirical observation, thus failing to understand practical robustness. To address this issue, we introduce Signal Embedding Energy (SEE), a method for quantifying the impact of noise intensity on LALM inputs, enabling the differentiation of LALM robustness in real-world deployments. SEE introduces a perspective based on structured activation subspaces derived from the model’s internal representations, which more accurately captures its perception of noise than raw audio features. Across experiments, SEE exhibits a strong correlation with LALM performance, achieving a correlation of 0.98. Surprisingly, traditional audio denoising methods are only marginally effective for LALMs, and, in some cases, even increase SEE and impair performance. This suggests a mismatch between speech-centric denoising objectives and the noise sensitivity of modern LALMs. Therefore, we propose a mitigation strategy derived from SEE to denoise LALM inputs, outperforming existing denoising methods. This paper introduces a novel metric for noise quantification in LALMs, providing guidance for robustness improvements in real-world deployments.
PAM: Enhancing General Alignment of Large Reasoning Models through Priority-Aware Metacognition
Zhihao Xu | Fuzhen Yang | Liang Lin | Xiting Wang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Zhihao Xu | Fuzhen Yang | Liang Lin | Xiting Wang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Recent advancements in Large Reasoning Models (LRMs) have showcased strong performance across various reasoning tasks by leveraging System-2 thinking capabilities. However, existing studies indicate that this reasoning ability alone does not reliably transfer to the general alignment domain. Inspired by cognitive science and how humans solve tasks, we argue that LRMs must be equipped with metacognitive knowledge to fully utilize their System-2 capabilities. In this paper, we propose Priority-Aware Metacognition (PAM), which guides the model to first identify the top-level human preference (e.g., harmlessness) as a means of understanding the alignment task’s nature, and then apply other kinds of metacognitive knowledge to better monitor and regulate the model’s thinking process. We implement PAM via a two-stage pipeline: a cold-start phase that collects structured metacognitive knowledge based on Flavell’s theoretical framework, and a preference-optimization phase that further reinforces such metacognition. Extensive experiments validate the effectiveness of PAM. Under the same training pipelines, PAM consistently yields higher performance, improving general domain alignment performance by ~10 points on the helpfulness and harmless benchmarks. Code is available at https://anonymous.4open.science/r/PAM-RM-02DF.
Visually-Guided Policy Optimization for Multimodal Reasoning
Zengbin Wang | Feng Xiong | Liang Lin | Xuecai Hu | Yong Wang | Yanlin Wang | Man Zhang | Xiangxiang Chu
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Zengbin Wang | Feng Xiong | Liang Lin | Xuecai Hu | Yong Wang | Yanlin Wang | Man Zhang | Xiangxiang Chu
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Reinforcement learning with verifiable rewards (RLVR) has significantly advanced the reasoning ability of vision-language models (VLMs). However, the inherent text-dominated nature of VLMs often leads to insufficient visual faithfulness, characterized by sparse attention activation to visual tokens. More importantly, our empirical analysis reveals that temporal visual forgetting along reasoning steps exacerbates this deficiency. To bridge this gap, we propose Visually-Guided Policy Optimization (VGPO), a novel framework to reinforce visual focus during policy optimization. Specifically, VGPO initially introduces a Visual Attention Compensation mechanism that leverages visual similarity to localize and amplify visual cues, while progressively elevating visual expectations in later steps to counteract visual forgetting. Building on this mechanism, we implement a dual-grained advantage re-weighting strategy: the intra-trajectory level highlights tokens exhibiting relatively high visual activation, while the inter-trajectory level prioritizes trajectories demonstrating superior visual accumulation. Extensive experiments demonstrate that VGPO achieves better visual activation and superior performance in mathematical multimodal reasoning and visual-dependent tasks. The code has been released at https://github.com/wzb-bupt/VGPO.
Search
Fix author
Co-authors
- Zhenhong Zhou 4
- Li Sun 3
- Kun Wang 3
- Moayad Aloqaily 2
- Yitian Chen 2
- Kaiwen Luo 2
- Jin Wang 2
- Qingsong Wen 2
- Shilinlu Yan 2
- Yibo Zhang 2
- Xiangxiang Chu 1
- Yaoqi Guo 1
- Xuecai Hu 1
- Prakhar Mehrotra 1
- Linsey Pang 1
- Yalan Qin 1
- Sen Su 1
- Xuehai Tang 1
- Jiayu Tian 1
- Weiliu Wang 1
- Xiting Wang 1
- Yanlin Wang 1
- Yong Wang 1
- Zengbin Wang 1
- Feng Xiong 1
- Zhihao Xu 1
- Fuzhen Yang 1
- Miao Yu 1
- Man Zhang 1
- Yuanhe Zhang 1