Pengyu Li
2026
AGTAO: Robust and Stabilized LLM Unlearning via Adversarial Gating Training with Adaptive Orthogonality
Pengyu Li | Lingling Zhang | Zhitao Gao | Yanrui Wu | Yuxuan Dong | Huan Liu | Bifan Wei | Jun Liu
Findings of the Association for Computational Linguistics: ACL 2026
Pengyu Li | Lingling Zhang | Zhitao Gao | Yanrui Wu | Yuxuan Dong | Huan Liu | Bifan Wei | Jun Liu
Findings of the Association for Computational Linguistics: ACL 2026
While Large Language Models (LLMs) have achieved remarkable capabilities, they unintentionally memorize sensitive data, posing critical privacy and security risks.Machine unlearning is pivotal for mitigating these risks, yet existing paradigms face a fundamental dilemma: aggressive unlearning often induces catastrophic forgetting that degrades model utility, whereas conservative strategies risk superficial forgetting, leaving models vulnerable to adversarial recovery. To address this trade-off, we propose AGTAO (Adversarial Gating Training with Adaptive Orthogonality), a unified framework designed to reconcile robust erasure with utility preservation. Specifically, our approach introduces Adaptive Orthogonality (AO) to dynamically mitigate geometric gradient conflicts between forgetting and retention objectives, thereby minimizing unintended knowledge degradation. Concurrently, Adversarial Gating Training (AGT) formulates unlearning as a latent-space min-max game, employing a curriculum-based gating mechanism to simulate and counter internal recovery attempts. Extensive experiments demonstrate that AGTAO achieves a superior trade-off between unlearning efficacy (KUR ≈ 0.01) and model utility (MMLU 58.30).[Code is available at <https://anonymous.4open.science/r/AGT-unlearning>.].
PhysPRM: A Generative Process Reward Model with Fine-grained Diagnosis for Physics Problem Solving
Yuxuan Dong | Xinyu Zhang | Lingling Zhang | Han Lai | Pengyu Li | Bifan Wei | Yaqiang Wu | Jun Liu
Findings of the Association for Computational Linguistics: ACL 2026
Yuxuan Dong | Xinyu Zhang | Lingling Zhang | Han Lai | Pengyu Li | Bifan Wei | Yaqiang Wu | Jun Liu
Findings of the Association for Computational Linguistics: ACL 2026
Despite the remarkable progress of Large Language Models (LLMs) in abstract reasoning tasks, they continue to struggle with physics problem solving due to difficulties in decoding implicit constraints and maintaining physical consistency. To address these challenges, Process Reward Models (PRMs) have emerged as a promising approach to verify intermediate reasoning steps. Existing PRMs attempt to mitigate reasoning errors but typically rely on scalar scoring, which lacks the explanatory power necessary to diagnose complex physical misconceptions. In this work, we introduce PhysPRM, a Generative PRM that treats evaluation as a generative task to produce fine-grained diagnoses comprising critiques, final judgments, and specific error types. To facilitate this, we develop an automated data synthesis pipeline to construct PhysPRM30K, a comprehensive training dataset, and PhysProcessBench, a rigorously human-verified benchmark. By employing a two-stage training paradigm that integrates Supervised Fine-Tuning with Group Relative Policy Optimization, PhysPRM significantly enhances the physics reasoning capabilities of various LLMs. Extensive experiments demonstrate that PhysPRM improves performance across seven benchmarks in both Best-of-N and critique refinement strategies.
2020
COVID-19 Literature Topic-Based Search via Hierarchical NMF
Rachel Grotheer | Longxiu Huang | Yihuan Huang | Alona Kryshchenko | Oleksandr Kryshchenko | Pengyu Li | Xia Li | Elizaveta Rebrova | Kyung Ha | Deanna Needell
Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020
Rachel Grotheer | Longxiu Huang | Yihuan Huang | Alona Kryshchenko | Oleksandr Kryshchenko | Pengyu Li | Xia Li | Elizaveta Rebrova | Kyung Ha | Deanna Needell
Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020
A dataset of COVID-19-related scientific literature is compiled, combining the articles from several online libraries and selecting those with open access and full text available. Then, hierarchical nonnegative matrix factorization is used to organize literature related to the novel coronavirus into a tree structure that allows researchers to search for relevant literature based on detected topics. We discover eight major latent topics and 52 granular subtopics in the body of literature, related to vaccines, genetic structure and modeling of the disease and patient studies, as well as related diseases and virology. In order that our tool may help current researchers, an interactive website is created that organizes available literature using this hierarchical structure.
2019
NeuralClassifier: An Open-source Neural Hierarchical Multi-label Text Classification Toolkit
Liqun Liu | Funan Mu | Pengyu Li | Xin Mu | Jing Tang | Xingsheng Ai | Ran Fu | Lifeng Wang | Xing Zhou
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations
Liqun Liu | Funan Mu | Pengyu Li | Xin Mu | Jing Tang | Xingsheng Ai | Ran Fu | Lifeng Wang | Xing Zhou
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations
In this paper, we introduce NeuralClassifier, a toolkit for neural hierarchical multi-label text classification. NeuralClassifier is designed for quick implementation of neural models for hierarchical multi-label classification task, which is more challenging and common in real-world scenarios. A salient feature is that NeuralClassifier currently provides a variety of text encoders, such as FastText, TextCNN, TextRNN, RCNN, VDCNN, DPCNN, DRNN, AttentiveConvNet and Transformer encoder, etc. It also supports other text classification scenarios, including binary-class and multi-class classification. Built on PyTorch, the core operations are calculated in batch, making the toolkit efficient with the acceleration of GPU. Experiments show that models built in our toolkit achieve comparable performance with reported results in the literature.