Vocabulary Pyramid Network: Multi-Pass Encoding and Decoding with Multi-Level Vocabularies for Response Generation

Cao Liu, Shizhu He, Kang Liu, Jun Zhao


Abstract
We study the task of response generation. Conventional methods employ a fixed vocabulary and one-pass decoding, which not only make them prone to safe and general responses but also lack further refining to the first generated raw sequence. To tackle the above two problems, we present a Vocabulary Pyramid Network (VPN) which is able to incorporate multi-pass encoding and decoding with multi-level vocabularies into response generation. Specifically, the dialogue input and output are represented by multi-level vocabularies which are obtained from hierarchical clustering of raw words. Then, multi-pass encoding and decoding are conducted on the multi-level vocabularies. Since VPN is able to leverage rich encoding and decoding information with multi-level vocabularies, it has the potential to generate better responses. Experiments on English Twitter and Chinese Weibo datasets demonstrate that VPN remarkably outperforms strong baselines.
Anthology ID:
P19-1367
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3774–3783
Language:
URL:
https://aclanthology.org/P19-1367
DOI:
10.18653/v1/P19-1367
Bibkey:
Cite (ACL):
Cao Liu, Shizhu He, Kang Liu, and Jun Zhao. 2019. Vocabulary Pyramid Network: Multi-Pass Encoding and Decoding with Multi-Level Vocabularies for Response Generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3774–3783, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Vocabulary Pyramid Network: Multi-Pass Encoding and Decoding with Multi-Level Vocabularies for Response Generation (Liu et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1367.pdf