On Incorporating Structural Information to improve Dialogue Response Generation

Nikita Moghe, Priyesh Vijayan, Balaraman Ravindran, Mitesh M. Khapra


Abstract
We consider the task of generating dialogue responses from background knowledge comprising of domain specific resources. Specifically, given a conversation around a movie, the task is to generate the next response based on background knowledge about the movie such as the plot, review, Reddit comments etc. This requires capturing structural, sequential and semantic information from the conversation context and the background resources. We propose a new architecture that uses the ability of BERT to capture deep contextualized representations in conjunction with explicit structure and sequence information. More specifically, we use (i) Graph Convolutional Networks (GCNs) to capture structural information, (ii) LSTMs to capture sequential information and (iii) BERT for the deep contextualized representations that capture semantic information. We analyze the proposed architecture extensively. To this end, we propose a plug-and-play Semantics-Sequences-Structures (SSS) framework which allows us to effectively combine such linguistic information. Through a series of experiments we make some interesting observations. First, we observe that the popular adaptation of the GCN model for NLP tasks where structural information (GCNs) was added on top of sequential information (LSTMs) performs poorly on our task. This leads us to explore interesting ways of combining semantic and structural information to improve the performance. Second, we observe that while BERT already outperforms other deep contextualized representations such as ELMo, it still benefits from the additional structural information explicitly added using GCNs. This is a bit surprising given the recent claims that BERT already captures structural information. Lastly, the proposed SSS framework gives an improvement of 7.95% on BLUE score over the baseline.
Anthology ID:
2020.nlp4convai-1.2
Volume:
Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI
Month:
July
Year:
2020
Address:
Online
Editors:
Tsung-Hsien Wen, Asli Celikyilmaz, Zhou Yu, Alexandros Papangelis, Mihail Eric, Anuj Kumar, Iñigo Casanueva, Rushin Shah
Venue:
NLP4ConvAI
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
11–24
Language:
URL:
https://aclanthology.org/2020.nlp4convai-1.2
DOI:
10.18653/v1/2020.nlp4convai-1.2
Bibkey:
Cite (ACL):
Nikita Moghe, Priyesh Vijayan, Balaraman Ravindran, and Mitesh M. Khapra. 2020. On Incorporating Structural Information to improve Dialogue Response Generation. In Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI, pages 11–24, Online. Association for Computational Linguistics.
Cite (Informal):
On Incorporating Structural Information to improve Dialogue Response Generation (Moghe et al., NLP4ConvAI 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.nlp4convai-1.2.pdf
Video:
 http://slideslive.com/38929628
Code
 nikitacs16/horovod_gcn_pointer_generator
Data
Holl-E