Pan Yang


2024

pdf bib
PEK: A Parameter-Efficient Framework for Knowledge-Grounded Dialogue Generation
Pan Yang | Dandan Song | Zhijing Wu | Yanru Zhou
Findings of the Association for Computational Linguistics: ACL 2024

Pre-trained language models (PLMs) have shown great dialogue generation capability in different scenarios. However, the huge VRAM consumption when fine-tuning them is one of their drawbacks. PEFT approaches can significantly reduce the number of trainable parameters, which enables us to fine-tune larger dialogue generation models. However, the reduction in parameter quantity can diminish a PLM’s expressive capacity and affect the PLM’s learning from certain specific examples like knowledge-related conversations. Previous works have demonstrated that injecting external knowledge into dialogue generation models can improve the model’s performance in knowledge-related conversations. Nonetheless, these methods are designed for the scenario where most parameters of the entire framework are trainable. In this paper, we propose PEK, a parameter-efficient framework for knowledge-enhanced dialogue generation. It enables PLMs to leverage external knowledge documents and knowledge graphs to enhance its generation capabilities with an acceptable number of trainable parameters. Evaluation results on the Wizard of Wikipedia and CMU_DoG datasets show that our approach outperforms baseline methods on multiple evaluation metrics, which validates the effectiveness of our approach.

pdf bib
KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extraction
Zixuan Li | Yutao Zeng | Yuxin Zuo | Weicheng Ren | Wenxuan Liu | Miao Su | Yucan Guo | Yantao Liu | Lixiang Lixiang | Zhilei Hu | Long Bai | Wei Li | Yidan Liu | Pan Yang | Xiaolong Jin | Jiafeng Guo | Xueqi Cheng
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

2021

pdf bib
Enhanced Language Representation with Label Knowledge for Span Extraction
Pan Yang | Xin Cong | Zhenyu Sun | Xingwu Liu
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

Span extraction, aiming to extract text spans (such as words or phrases) from plain text, is a fundamental process in Information Extraction. Recent works introduce the label knowledge to enhance the text representation by formalizing the span extraction task into a question answering problem (QA Formalization), which achieves state-of-the-art performance. However, such a QA Formalization does not fully exploit the label knowledge and causes a dramatic decrease in efficiency of training/inference. To address those problems, we introduce a fresh paradigm to integrate label knowledge and further propose a novel model to explicitly and efficiently integrate label knowledge into text representations. Specifically, it encodes texts and label annotations independently and then integrates label knowledge into text representation with an elaborate-designed semantics fusion module. We conduct extensive experiments on three typical span extraction tasks: flat NER, nested NER, and event detection. The empirical results show that 1) our model achieves a new state-of-the-art performance on four benchmarks, and 2) reduces training time and inference time by 76% and 77% on average, respectively, compared with the QA Formalization paradigm.