Large Scale Multi-Lingual Multi-Modal Summarization Dataset

Yash Verma, Anubhav Jangra, Raghvendra Verma, Sriparna Saha


Abstract
Significant developments in techniques such as encoder-decoder models have enabled us to represent information comprising multiple modalities. This information can further enhance many downstream tasks in the field of information retrieval and natural language processing; however, improvements in multi-modal techniques and their performance evaluation require large-scale multi-modal data which offers sufficient diversity. Multi-lingual modeling for a variety of tasks like multi-modal summarization, text generation, and translation leverages information derived from high-quality multi-lingual annotated data. In this work, we present the current largest multi-lingual multi-modal summarization dataset (M3LS), and it consists of over a million instances of document-image pairs along with a professionally annotated multi-modal summary for each pair. It is derived from news articles published by British Broadcasting Corporation(BBC) over a decade and spans 20 languages, targeting diversity across five language roots, it is also the largest summarization dataset for 13 languages and consists of cross-lingual summarization data for 2 languages. We formally define the multi-lingual multi-modal summarization task utilizing our dataset and report baseline scores from various state-of-the-art summarization techniques in a multi-lingual setting. We also compare it with many similar datasets to analyze the uniqueness and difficulty of M3LS. The dataset and code used in this work are made available at “https://github.com/anubhav-jangra/M3LS”.
Anthology ID:
2023.eacl-main.263
Volume:
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3620–3632
Language:
URL:
https://aclanthology.org/2023.eacl-main.263
DOI:
10.18653/v1/2023.eacl-main.263
Bibkey:
Cite (ACL):
Yash Verma, Anubhav Jangra, Raghvendra Verma, and Sriparna Saha. 2023. Large Scale Multi-Lingual Multi-Modal Summarization Dataset. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3620–3632, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
Large Scale Multi-Lingual Multi-Modal Summarization Dataset (Verma et al., EACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.eacl-main.263.pdf
Video:
 https://aclanthology.org/2023.eacl-main.263.mp4