Fuchun Peng


2021

pdf bib
Analyzing the Forgetting Problem in Pretrain-Finetuning of Open-domain Dialogue Response Models
Tianxing He | Jun Liu | Kyunghyun Cho | Myle Ott | Bing Liu | James Glass | Fuchun Peng
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume

In this work, we study how the finetuning stage in the pretrain-finetune framework changes the behavior of a pretrained neural language generator. We focus on the transformer encoder-decoder model for the open-domain dialogue response generation task. Our major finding is that after standard finetuning, the model forgets some of the important language generation skills acquired during large-scale pretraining. We demonstrate the forgetting phenomenon through a set of detailed behavior analysis from the perspectives of knowledge transfer, context sensitivity, and function space projection. As a preliminary attempt to alleviate the forgetting problem, we propose an intuitive finetuning strategy named “mix-review”. We find that mix-review effectively regularizes the finetuning process, and the forgetting problem is alleviated to some extent. Finally, we discuss interesting behavior of the resulting dialogue model and its implications.

2010

pdf bib
Search with Synonyms: Problems and Solutions
Xing Wei | Fuchun Peng | Huishin Tseng | Yumao Lu | Xuerui Wang | Benoit Dumoulin
Coling 2010: Posters

2009

pdf bib
Improving Web Search Relevance with Semantic Features
Yumao Lu | Fuchun Peng | Gilad Mishne | Xing Wei | Benoit Dumoulin
Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing

2006

pdf bib
Chinese Named Entity Recognition with Conditional Probabilistic Models
Aitao Chen | Fuchun Peng | Roy Shan | Gordon Sun
Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing

2005

pdf bib
Combining Deep Linguistics Analysis and Surface Pattern Learning: A Hybrid Approach to Chinese Definitional Question Answering
Fuchun Peng | Ralph Weischedel | Ana Licuanan | Jinxi Xu
Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing

2004

pdf bib
Chinese Segmentation and New Word Detection using Conditional Random Fields
Fuchun Peng | Fangfang Feng | Andrew McCallum
COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics

pdf bib
Accurate Information Extraction from Research Papers using Conditional Random Fields
Fuchun Peng | Andrew McCallum
Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004

2003

pdf bib
Language Independent Authorship Attribution with Character Level N-Grams
Fuchun Peng | Dale Schuurmans | Vlado Keselj | Shaojun Wang
10th Conference of the European Chapter of the Association for Computational Linguistics

pdf bib
Text Classification in Asian Languages without Word Segmentation
Fuchun Peng | Xiangji Huang | Dale Schuurmans | Shaojun Wang
Proceedings of the Sixth International Workshop on Information Retrieval with Asian Languages

pdf bib
Language and Task Independent Text Categorization with Simple Language Models
Fuchun Peng | Dale Schuurmans | Shaojun Wang
Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics

2002

pdf bib
Investigating the Relationship between Word Segmentation Performance and Retrieval Performance in Chinese IR
Fuchun Peng | Xiangji Huang | Dale Schuurmans | Nick Cercone
COLING 2002: The 19th International Conference on Computational Linguistics