Balancing Multi-Domain Corpora Learning for Open-Domain Response Generation

Yujie Xing, Jinglun Cai, Nils Barlaug, Peng Liu, Jon Atle Gulla


Abstract
Open-domain conversational systems are assumed to generate equally good responses on multiple domains. Previous work achieved good performance on the single corpus, but training and evaluating on multiple corpora from different domains are less studied. This paper explores methods of generating relevant responses for each of multiple multi-domain corpora. We first examine interleaved learning which intermingles multiple corpora as the baseline. We then investigate two multi-domain learning methods, labeled learning and multi-task labeled learning, which encode each corpus through a unique corpus embedding. Furthermore, we propose Domain-specific Frequency (DF), a novel word-level importance weight that measures the relative importance of a word for a specific corpus compared to other corpora. Based on DF, we propose weighted learning, a method that integrates DF to the loss function. We also adopt DF as a new evaluation metric. Extensive experiments show that our methods gain significant improvements on both automatic and human evaluation. We share our code and data for reproducibility.
Anthology ID:
2022.findings-naacl.162
Volume:
Findings of the Association for Computational Linguistics: NAACL 2022
Month:
July
Year:
2022
Address:
Seattle, United States
Editors:
Marine Carpuat, Marie-Catherine de Marneffe, Ivan Vladimir Meza Ruiz
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2104–2120
Language:
URL:
https://aclanthology.org/2022.findings-naacl.162
DOI:
10.18653/v1/2022.findings-naacl.162
Bibkey:
Cite (ACL):
Yujie Xing, Jinglun Cai, Nils Barlaug, Peng Liu, and Jon Atle Gulla. 2022. Balancing Multi-Domain Corpora Learning for Open-Domain Response Generation. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 2104–2120, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
Balancing Multi-Domain Corpora Learning for Open-Domain Response Generation (Xing et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-naacl.162.pdf
Software:
 2022.findings-naacl.162.software.zip
Video:
 https://aclanthology.org/2022.findings-naacl.162.mp4
Data
ConvAI2