Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization

Xiachong Feng, Xiaocheng Feng, Libo Qin, Bing Qin, Ting Liu


Abstract
Current dialogue summarization systems usually encode the text with a number of general semantic features (e.g., keywords and topics) to gain more powerful dialogue modeling capabilities. However, these features are obtained via open-domain toolkits that are dialog-agnostic or heavily relied on human annotations. In this paper, we show how DialoGPT, a pre-trained model for conversational response generation, can be developed as an unsupervised dialogue annotator, which takes advantage of dialogue background knowledge encoded in DialoGPT. We apply DialoGPT to label three types of features on two dialogue summarization datasets, SAMSum and AMI, and employ pre-trained and non pre-trained models as our summarizers. Experimental results show that our proposed method can obtain remarkable improvements on both datasets and achieves new state-of-the-art performance on the SAMSum dataset.
Anthology ID:
2021.acl-long.117
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
August
Year:
2021
Address:
Online
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1479–1491
Language:
URL:
https://aclanthology.org/2021.acl-long.117
DOI:
10.18653/v1/2021.acl-long.117
Bibkey:
Cite (ACL):
Xiachong Feng, Xiaocheng Feng, Libo Qin, Bing Qin, and Ting Liu. 2021. Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1479–1491, Online. Association for Computational Linguistics.
Cite (Informal):
Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization (Feng et al., ACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-long.117.pdf
Video:
 https://aclanthology.org/2021.acl-long.117.mp4
Code
 xcfcode/PLM_annotator
Data
SAMSum Corpus