Plug-and-Play Adaptation for Continuously-updated QA

Kyungjae Lee, Wookje Han, Seung-won Hwang, Hwaran Lee, Joonsuk Park, Sang-Woo Lee


Abstract
Language models (LMs) have shown great potential as implicit knowledge bases (KBs). And for their practical use, knowledge in LMs need to be updated periodically. However, existing tasks to assess LMs’ efficacy as KBs do not adequately consider multiple large-scale updates. To this end, we first propose a novel task—Continuously-updated QA (CuQA)—in which multiple large-scale updates are made to LMs, and the performance is measured with respect to the success in adding and updating knowledge while retaining existing knowledge. We then present LMs with plug-in modules that effectively handle the updates. Experiments conducted on zsRE QA and NQ datasets show that our method outperforms existing approaches. We find that our method is 4x more effective in terms of updates/forgets ratio, compared to a fine-tuning baseline.
Anthology ID:
2022.findings-acl.37
Volume:
Findings of the Association for Computational Linguistics: ACL 2022
Month:
May
Year:
2022
Address:
Dublin, Ireland
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
438–447
Language:
URL:
https://aclanthology.org/2022.findings-acl.37
DOI:
10.18653/v1/2022.findings-acl.37
Bibkey:
Cite (ACL):
Kyungjae Lee, Wookje Han, Seung-won Hwang, Hwaran Lee, Joonsuk Park, and Sang-Woo Lee. 2022. Plug-and-Play Adaptation for Continuously-updated QA. In Findings of the Association for Computational Linguistics: ACL 2022, pages 438–447, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Plug-and-Play Adaptation for Continuously-updated QA (Lee et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-acl.37.pdf
Software:
 2022.findings-acl.37.software.zip
Data
Natural QuestionsSituatedQA