Global Voices: Crossing Borders in Automatic News Summarization

Khanh Nguyen, Hal Daumé III


Abstract
We construct Global Voices, a multilingual dataset for evaluating cross-lingual summarization methods. We extract social-network descriptions of Global Voices news articles to cheaply collect evaluation data for into-English and from-English summarization in 15 languages. Especially, for the into-English summarization task, we crowd-source a high-quality evaluation dataset based on guidelines that emphasize accuracy, coverage, and understandability. To ensure the quality of this dataset, we collect human ratings to filter out bad summaries, and conduct a survey on humans, which shows that the remaining summaries are preferred over the social-network summaries. We study the effect of translation quality in cross-lingual summarization, comparing a translate-then-summarize approach with several baselines. Our results highlight the limitations of the ROUGE metric that are overlooked in monolingual summarization.
Anthology ID:
D19-5411
Volume:
Proceedings of the 2nd Workshop on New Frontiers in Summarization
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Lu Wang, Jackie Chi Kit Cheung, Giuseppe Carenini, Fei Liu
Venue:
WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
90–97
Language:
URL:
https://aclanthology.org/D19-5411
DOI:
10.18653/v1/D19-5411
Bibkey:
Cite (ACL):
Khanh Nguyen and Hal Daumé III. 2019. Global Voices: Crossing Borders in Automatic News Summarization. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, pages 90–97, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Global Voices: Crossing Borders in Automatic News Summarization (Nguyen & Daumé III, 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-5411.pdf
Data
Global Voices