Guoshuai Zhao

Author directory

2026

Large language models (LLMs) encode vast amounts of knowledge in their parameters, but these facts often require precise post-training rectification as they become outdated or incorrect over time. Traditional batch editing methods typically apply a uniform update strength across all instances. However, this uniform approach ignores the inherent variation in editing difficulty, leading to a fundamental trade-off between correction success and model stability. To overcome this limitation, we introduce AdaPE (Adaptive Pressure Editing), a parameter-free adaptive editing framework that dynamically adjusts the update behavior for each specific instance. Specifically, we derive a closed-form adaptive update rate based on the ratio of residual energy to spatial sensitivity, which strictly quantifies the intrinsic injection difficulty. This formulation adaptively limits the update strength for difficult facts to preserve model locality, while ensuring sufficient knowledge injection for easier ones. Extensive experiments on three benchmarks and two prominent LLMs demonstrate the superiority of our approach in consistently circumventing the fixed-strength bottleneck, achieving a new state-of-the-art balance between editing efficacy and model integrity. Our code is available at https://github.com/skyandkibo/adape.
Multi-agent LLM systems decompose complex tasks across specialist agents, yet joint prompt optimization remains difficult because agents are coupled through structured information handoffs. We identify interface mismatch as a structural failure mode that existing optimizers cannot diagnose: their scalar attribution signals conflate reasoning quality with format incompatibility, and the inter-agent contract is left entirely implicit. We propose BID-SPO (Bidding-based Shapley Prompt Optimization), which makes the inter-agent contract an explicit optimization variable. Each agent declares an Output Bid and an Input Demand, and a contract-aware cooperative-game characteristic function yields an attribution signal whose Shapley decomposition exposes three interpretable dimensions (global task contribution, downstream interface alignment, and upstream adaptation cost), turning attribution into a targeted diagnosis for prompt repair. A bid-conformance constraint keeps declared contracts feasible with respect to actual agent behavior throughout optimization. Across seven benchmarks and different model configurations, BID-SPO outperforms MASPO and AgentDropout, with accuracy gains of +2.93% under the Qwen3-8B + Gemini-2.5-pro configuration and +4.38% under the DeepSeek-v4 flash / pro configuration. Our findings emphasize that reliable multi-agent LLM optimization requires optimizing inter-agent interfaces as explicit objects, not just improving agents in isolation.
Large language models exhibit significant potential for psychological support, yet they often generate fragmented and emotionally inconsistent dialogues that lack the therapeutic structure necessary for reliable assessment.To address these issues, we introduce VeilEval, a clinically grounded and privacy-preserving benchmark equipped with interpretable metrics for evaluating multi-turn psychological dialogues.Furthermore, we propose Emotion-Resonance (EmoRes), a multi-agent framework that boosts psychological reasoning via a Topic-Mining Emotional Agent and a multi-perspective Self-Reflection Agent, thereby jointly improving topic continuity, emotional coherence, and clinical interpretability.Experiments demonstrate that EmoRes achieves up to ∼ 3× improvement over strong baselines on VeilEval, with its effectiveness further validated by ablation studies and human evaluations.
The increasing integration of large language models (LLMs) in code generation has raised critical copyright concerns, particularly regarding the verbatim repetition of copyrighted code. To address this challenge, we propose a novel task: Duplicate-Aware Controlled Code Generation (DACCG), which aims to mitigate verbatim repetition while preserving the quality of generated code. To this end, we introduce Targeted Reordering Beam Search (TRBS), a plug-and-play decoding method that dynamically reorders beam candidates to reduce direct copying. TRBS leverages the FM-index for efficient substring detection and employs a spike-entropy-based protection mechanism to safeguard structural anchors critical to code coherence. Experimental results on a multi-language code generation benchmark demonstrate that TRBS effectively reduces verbatim repetition while maintaining functional adequacy. Our research represents a pioneering effort in code copyright protection from the model user’s perspective, offering novel insights into responsible code generation practices.
Multi-hop Question Answering (MHQA) aims to answer questions that require multi-step reasoning. The complexity of user queries, coupled with potential knowledge deficiencies in Large Language Models (LLMs), gives rise to two pivotal challenges that underpin the performance on this task: the correct identification of the reasoning path and the accurate retrieval of essential knowledge. Existing approaches primarily rely on prompt-based methods to generate reasoning paths, which are further combined with traditional sparse or dense retrieval to produce the final answer. However, the generation of reasoning paths commonly lacks effective control over the generative process, thus leading the reasoning astray. Meanwhile, the retrieval methods over-rely on knowledge matching or similarity scores rather than evaluating the practical utility of the information, resulting in retrieving homogeneous or non-useful information. Therefore, we propose a Structured Entity-Aware Retrieval with Chain-of-Reasoning Navigator framework named SEARCH-R. Specifically, SEARCH-R trains an end-to-end reasoning path navigator, which is able to provide a powerful sub-question decomposer by fine-tuning the Llama3.1-8B model. Moreover, a novel dependency tree-based retrieval is designed to evaluate the informational contribution of the document quantitatively. Extensive experiments on three challenging multi-hop datasets validate the effectiveness of the proposed framework. The code and dataset are available at: https://github.com/Applied-Machine-Learning-Lab/ACL2026_SEARCH-R.
Diagnostic prediction and clinical reasoning are critical tasks in healthcare applications. While large language models have shown strong capabilities in commonsense reasoning, they still struggle with diagnostic reasoning due to limited domain knowledge. Existing approaches often rely on internal model knowledge or static knowledge bases, which are insufficient to support the knowledge demands of diagnostic reasoning. Moreover, these methods focus solely on the accuracy of final predictions, overlooking alignment with standard clinical reasoning trajectories. To this end, we propose MultiDx, a two-stage diagnostic reasoning framework that performs differential diagnosis by analyzing evidence collected from multiple knowledge sources. Specifically, it first generates suspected diagnoses and reasoning traces by leveraging knowledge from web search, SOAP-formatted case, and clinical case database. Then it integrates multi-perspective evidence through matching, voting, and differential diagnosis to generate the final prediction. Extensive experiments demonstrate the effectiveness of our approach.
Large language models have demonstrated strong reasoning capabilities in general knowledge question answering. However, their ability to handle temporal information remains limited. To address this limitation, existing approaches often design time-sensitive reasoning pipelines that rely on external tools or manual verification and are tailored to specific scenarios, leading to poor generalizability. Moreover, these methods apply a fixed pipeline to all questions, overlooking the fact that different types of temporal questions often require distinct reasoning strategies, which leads to unnecessary processing for simple cases and inadequate reasoning for more complex ones. To this end, we propose AdapTime, an adaptive temporal reasoning method that dynamically executes reasoning steps based on the input context and task requirements. Specifically, it involves three temporal reasoning actions: reformulate, rewrite and review, with an LLM planner guiding the reasoning process. AdapTime integrates seamlessly with state-of-the-art LLMs and significantly enhances their temporal reasoning capabilities without relying on external support. Extensive experiments on two temporal QA benchmarks demonstrate the effectiveness of our approach.
Reasoning in Large Language Models (LLMs) is primarily conducted in a discrete token space. Although recent approaches employing continuous concept tokens facilitate abstract reasoning, they introduce significant stability challenges. The accumulation of Out-of-Distribution (OOD) states frequently causes manifold deviation, while the absence of clear boundaries leads to indeterminate termination, often referred to as the anchoring problem. To address these limitations, we propose HybridThink. We formulate mode selection in LLM reasoning as a sequential decision process. A lightweight gating policy is trained with RL to decide at each step whether to propagate a continuous concept token or generate a discrete text token to anchor the semantic trajectory.This learned interleaving limits manifold deviation and resolves the fragility of static rules. Furthermore, we ensure stable convergence through a training strategy that incorporates Error-Centric Experience Replay (ECER) and Inductive Exploration Shaping (IES), prioritizing learning from failure modes, specifically drifting and premature termination. Empirical evaluations on mathematics and coding benchmarks demonstrate that HybridThink outperforms standard CoT and Soft Thinking baselines, achieving superior accuracy with reduced token consumption.
Continuous-space reasoning methods replace discrete intermediate tokens with probability-weighted soft embeddings, aiming to preserve semantic information that greedy Chain-of-Thought decoding discards. We show that this promise is limited by local representation geometry and the latent semantic manifold. When the next-token distribution is low-entropy and concentrated, a soft token becomes nearly collinear with the Top-1 embedding and collapses toward greedy decoding. When the distribution is high-entropy, diffuse, or stochastically perturbed, the continuous state may leave the greedy path but drift away from the local scale of naturally occurring candidate representations. We diagnose these two regimes with step-level probes over representation divergence, local norm calibration, and behavior-cloud membership. To address them, we propose Manifold Thinking, a training-free inference framework that combines structured tangent movement away from the Top-1 direction with local norm-calibrated projection back to the candidate-token neighborhood. Across mathematical and programming benchmarks, Manifold Thinking improves over discrete CoT baselines and baselines covering the latest stochastic Soft Thinking strategies, suggesting that useful continuous reasoning requires geometric correction rather than unconstrained linear token mixing alone.
The rapid development of large language models (LLMs) has created an urgent need for identifying machine-generated texts, and text watermarking technology has proven to be an effective solution. However, current watermarking methods, while demonstrating strong detectability, significantly degrade text quality due to the introduction of unnatural tokens. The main reason lies in the fact that these methods ignore the importance of semantic information in the watermarking process. To address this issue, we note that the logit vector produced by LLMs encodes both semantic understanding of input texts and prediction confidence across different tokens. Therefore, we propose a novel Semantic Self-Guided Watermarking (SSGW) framework that leverages the LLM itself to generate a guidance logic vector that assists in watermarking while producing the original one concurrently. Subsequently, we design a transform module to analyze these two vectors comprehensively and then transform them into adaptive watermark logits for different candidate tokens, thereby reducing the possibility of selecting inappropriate tokens. Experimental results confirm the effectiveness of our method in achieving superior performance in both watermark detectability and text quality preservation. The source code will be made publicly available upon acceptance.

2025

Temporal knowledge graph reasoning aims to predict future events with knowledge of existing facts and plays a key role in various downstream tasks. Previous methods focused on either graph structure learning or semantic reasoning, failing to integrate dual reasoning perspectives to handle different prediction scenarios. Moreover, they lack the capability to capture the inherent differences between historical and non-historical events, which limits their generalization across different temporal contexts. To this end, we propose a Multi-Expert Structural-Semantic Hybrid (MESH) framework that employs three kinds of expert modules to integrate both structural and semantic information, guiding the reasoning process for different events. Extensive experiments on three datasets demonstrate the effectiveness of our approach.

2024

New intent discovery is a crucial capability for task-oriented dialogue systems. Existing methods focus on transferring in-domain (IND) prior knowledge to out-of-domain (OOD) data through pre-training and clustering stages. They either handle the two processes in a pipeline manner, which exhibits a gap between intent representation and clustering process or use typical contrastive clustering that overlooks the potential supervised signals from the whole data. Besides, they often deal with either open intent discovery or OOD settings individually. To this end, we propose a Pseudo-Label enhanced Prototypical Contrastive Learning (PLPCL) model for uniformed intent discovery. We iteratively utilize pseudo-labels to explore potential positive/negative samples for contrastive learning and bridge the gap between representation and clustering. To enable better knowledge transfer, we design a prototype learning method integrating the supervised and pseudo signals from IND and OOD samples. In addition, our method has been proven effective in two different settings of discovering new intents. Experiments on three benchmark datasets and two task settings demonstrate the effectiveness of our approach.
Large Language Models (LLMs) exhibit the issue of paraphrase divergence. This means that when a question is phrased in a slightly different but semantically similar way, LLM may output a wrong response despite being able to answer the original question correctly. Previous research has regarded this issue as a problem of the model’s robustness to question paraphrase and proposed a retraining method to address it. However, retraining faces challenges in meeting the computational costs and privacy security demands of LLMs. In this paper, we regard this issue as a problem of alignment with model preferences and propose PEARL (Preference-drivEn pAraphRase Learning). This is a black-box method that enhances model performance by paraphrasing questions in expressions preferred by the model. We validate PEARL across six datasets spanning three tasks: open-domain QA, commonsense reasoning, and math word problem. Extensive experiments demonstrated not only the outstanding performance but also the composability, transferability, and immense potential of PEARL, shedding new light on the black-box tuning of LLMs.