Pengyu Li
Author directoryOther people with similar names: Pengyu Li
Unverified author pages with similar names: Pengyu Li
2026
AGTAO: Robust and Stabilized LLM Unlearning via Adversarial Gating Training with Adaptive Orthogonality
Pengyu Li | Lingling Zhang | Zhitao Gao | Yanrui Wu | Yuxuan Dong | Huan Liu | Bifan Wei | Jun Liu
Findings of the Association for Computational Linguistics: ACL 2026
Pengyu Li | Lingling Zhang | Zhitao Gao | Yanrui Wu | Yuxuan Dong | Huan Liu | Bifan Wei | Jun Liu
Findings of the Association for Computational Linguistics: ACL 2026
While Large Language Models (LLMs) have achieved remarkable capabilities, they unintentionally memorize sensitive data, posing critical privacy and security risks.Machine unlearning is pivotal for mitigating these risks, yet existing paradigms face a fundamental dilemma: aggressive unlearning often induces catastrophic forgetting that degrades model utility, whereas conservative strategies risk superficial forgetting, leaving models vulnerable to adversarial recovery. To address this trade-off, we propose AGTAO (Adversarial Gating Training with Adaptive Orthogonality), a unified framework designed to reconcile robust erasure with utility preservation. Specifically, our approach introduces Adaptive Orthogonality (AO) to dynamically mitigate geometric gradient conflicts between forgetting and retention objectives, thereby minimizing unintended knowledge degradation. Concurrently, Adversarial Gating Training (AGT) formulates unlearning as a latent-space min-max game, employing a curriculum-based gating mechanism to simulate and counter internal recovery attempts. Extensive experiments demonstrate that AGTAO achieves a superior trade-off between unlearning efficacy (KUR ≈ 0.01) and model utility (MMLU 58.30).[Code is available at https://anonymous.4open.science/r/AGT-unlearning>.].
PhysPRM: A Generative Process Reward Model with Fine-grained Diagnosis for Physics Problem Solving
Yuxuan Dong | Xinyu Zhang | Lingling Zhang | Han Lai | Pengyu Li | Bifan Wei | Yaqiang Wu | Jun Liu
Findings of the Association for Computational Linguistics: ACL 2026
Yuxuan Dong | Xinyu Zhang | Lingling Zhang | Han Lai | Pengyu Li | Bifan Wei | Yaqiang Wu | Jun Liu
Findings of the Association for Computational Linguistics: ACL 2026
Despite the remarkable progress of Large Language Models (LLMs) in abstract reasoning tasks, they continue to struggle with physics problem solving due to difficulties in decoding implicit constraints and maintaining physical consistency. To address these challenges, Process Reward Models (PRMs) have emerged as a promising approach to verify intermediate reasoning steps. Existing PRMs attempt to mitigate reasoning errors but typically rely on scalar scoring, which lacks the explanatory power necessary to diagnose complex physical misconceptions. In this work, we introduce PhysPRM, a Generative PRM that treats evaluation as a generative task to produce fine-grained diagnoses comprising critiques, final judgments, and specific error types. To facilitate this, we develop an automated data synthesis pipeline to construct PhysPRM30K, a comprehensive training dataset, and PhysProcessBench, a rigorously human-verified benchmark. By employing a two-stage training paradigm that integrates Supervised Fine-Tuning with Group Relative Policy Optimization, PhysPRM significantly enhances the physics reasoning capabilities of various LLMs. Extensive experiments demonstrate that PhysPRM improves performance across seven benchmarks in both Best-of-N and critique refinement strategies.
LogicGraph: Benchmarking Multi-Path Logical Reasoning via Neuro-Symbolic Generation and Verification
Yanrui Wu | Lingling Zhang | Xinyu Zhang | Jiayu Chang | Pengyu Li | Xu Jiang | Jingtao Hu | Jun Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yanrui Wu | Lingling Zhang | Xinyu Zhang | Jiayu Chang | Pengyu Li | Xu Jiang | Jingtao Hu | Jun Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Evaluations of large language models (LLMs) primarily emphasize convergent logical reasoning, where success is defined by producing a single correct proof. However, many real-world reasoning problems admit multiple valid derivations, requiring models to explore diverse logical paths rather than committing to one route. To address this limitation, we introduce LogicGraph, the first benchmark aimed to systematically evaluate multi-path logical reasoning, constructed via a neuro-symbolic framework that leverages backward logic generation and semantic instantiation. This pipeline yields solver-verified reasoning problems formalized by high-depth multi-path reasoning and inherent logical distractions, where each instance is associated with an exhaustive set of minimal proofs. We further propose a reference-free evaluation framework to rigorously assess model performance in both convergent and divergent regimes. Experiments on state-of-the-art language models reveal a common limitation: models tend to commit early to a single route and fail to explore alternatives, and the coverage gap grows substantially with reasoning depth. LogicGraph exposes this divergence gap and provides actionable insights to motivate future improvements. Our code and data will be published at https://github.com/kkkkarry/LogicGraph/.