Improving Long Dialogue Summarization with Semantic Graph Representation

Yilun Hua, Zhaoyuan Deng, Kathleen McKeown


Abstract
Although Large Language Models (LLMs) are successful in abstractive summarization of short dialogues, summarization of long dialogues remains challenging. To address this challenge, we propose a novel algorithm that processes complete dialogues comprising thousands of tokens into topic-segment-level Abstract Meaning Representation (AMR) graphs, which explicitly capture the dialogue structure, highlight salient semantics, and preserve high-level information. We also develop a new text-graph attention to leverage both graph semantics and a pretrained LLM that exploits the text. Finally, we propose an AMR node selection loss used jointly with conventional cross-entropy loss, to create additional training signals that facilitate graph feature encoding and content selection. Experiments show that our system outperforms the state-of-the-art models on multiple long dialogue summarization datasets, especially in low-resource settings, and generalizes well to out-of-domain data.
Anthology ID:
2023.findings-acl.871
Volume:
Findings of the Association for Computational Linguistics: ACL 2023
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
13851–13883
Language:
URL:
https://aclanthology.org/2023.findings-acl.871
DOI:
10.18653/v1/2023.findings-acl.871
Bibkey:
Cite (ACL):
Yilun Hua, Zhaoyuan Deng, and Kathleen McKeown. 2023. Improving Long Dialogue Summarization with Semantic Graph Representation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 13851–13883, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Improving Long Dialogue Summarization with Semantic Graph Representation (Hua et al., Findings 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.findings-acl.871.pdf