@inproceedings{zhu-etal-2022-parameter,
    title = "Parameter-efficient Continual Learning Framework in Industrial Real-time Text Classification System",
    author = "Zhu, Tao  and
      Zhao, Zhe  and
      Liu, Weijie  and
      Liu, Jiachi  and
      Chen, Yiren  and
      Mao, Weiquan  and
      Liu, Haoyan  and
      Ding, Kunbo  and
      Li, Yudong  and
      Yang, Xuefeng",
    booktitle = "Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track",
    month = jul,
    year = "2022",
    address = "Hybrid: Seattle, Washington + Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.naacl-industry.35",
    doi = "10.18653/v1/2022.naacl-industry.35",
    pages = "315--323",
    abstract = "Catastrophic forgetting is a challenge for model deployment in industrial real-time systems, which requires the model to quickly master a new task without forgetting the old one. Continual learning aims to solve this problem; however, it usually updates all the model parameters, resulting in extensive training times and the inability to deploy quickly. To address this challenge, we propose a parameter-efficient continual learning framework, in which efficient parameters are selected through an offline parameter selection strategy and then trained using an online regularization method. In our framework, only a few parameters need to be updated, which not only alleviates catastrophic forgetting, but also allows the model to be saved with the changed parameters instead of all parameters. Extensive experiments are conducted to examine the effectiveness of our proposal. We believe this paper will provide useful insights and experiences on developing deep learning-based online real-time systems.",
}