DialogSum Challenge: Results of the Dialogue Summarization Shared Task

Yulong Chen, Naihao Deng, Yang Liu, Yue Zhang


Abstract
We report the results of DialogSum Challenge, the shared task on summarizing real-life sce- nario dialogues at INLG 2022. Four teams participate in this shared task and three submit their system reports, exploring different meth- ods to improve the performance of dialogue summarization. Although there is a great im- provement over the baseline models regarding automatic evaluation metrics, such as ROUGE scores, we find that there is a salient gap be- tween model generated outputs and human an- notated summaries by human evaluation from multiple aspects. These findings demonstrate the difficulty of dialogue summarization and suggest that more fine-grained evaluatuion met- rics are in need.
Anthology ID:
2022.inlg-genchal.14
Volume:
Proceedings of the 15th International Conference on Natural Language Generation: Generation Challenges
Month:
July
Year:
2022
Address:
Waterville, Maine, USA and virtual meeting
Editors:
Samira Shaikh, Thiago Ferreira, Amanda Stent
Venue:
INLG
SIG:
SIGGEN
Publisher:
Association for Computational Linguistics
Note:
Pages:
94–103
Language:
URL:
https://aclanthology.org/2022.inlg-genchal.14
DOI:
Bibkey:
Cite (ACL):
Yulong Chen, Naihao Deng, Yang Liu, and Yue Zhang. 2022. DialogSum Challenge: Results of the Dialogue Summarization Shared Task. In Proceedings of the 15th International Conference on Natural Language Generation: Generation Challenges, pages 94–103, Waterville, Maine, USA and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
DialogSum Challenge: Results of the Dialogue Summarization Shared Task (Chen et al., INLG 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.inlg-genchal.14.pdf