Pengxiang Cheng


pdf bib
Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning
Genta Winata | Lingjue Xie | Karthik Radhakrishnan | Shijie Wu | Xisen Jin | Pengxiang Cheng | Mayank Kulkarni | Daniel Preotiuc-Pietro
Findings of the Association for Computational Linguistics: ACL 2023

Real-life multilingual systems should be able to efficiently incorporate new languages as data distributions fed to the system evolve and shift over time. To do this, systems need to handle the issue of catastrophic forgetting, where the model performance drops for languages or tasks seen further in its past. In this paper, we study catastrophic forgetting, as well as methods to minimize this, in a massively multilingual continual learning framework involving up to 51 languages and covering both classification and sequence labeling tasks. We present LR ADJUST, a learning rate scheduling method that is simple, yet effective in preserving new information without strongly overwriting past knowledge. Furthermore, we show that this method is effective across multiple continual learning approaches. Finally, we provide further insights into the dynamics of catastrophic forgetting in this massively multilingual setup.


pdf bib
Implicit Argument Prediction with Event Knowledge
Pengxiang Cheng | Katrin Erk
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)

Implicit arguments are not syntactically connected to their predicates, and are therefore hard to extract. Previous work has used models with large numbers of features, evaluated on very small datasets. We propose to train models for implicit argument prediction on a simple cloze task, for which data can be generated automatically at scale. This allows us to use a neural model, which draws on narrative coherence and entity salience for predictions. We show that our model has superior performance on both synthetic and natural data.


pdf bib
Representing Meaning with a Combination of Logical and Distributional Models
I. Beltagy | Stephen Roller | Pengxiang Cheng | Katrin Erk | Raymond J. Mooney
Computational Linguistics, Volume 42, Issue 4 - December 2016