Automatic Summarization of Long Documents

Naman Chhibbar, Jugal Kalita


Abstract
A vast amount of textual data is added to the internet daily, making utilization and interpretation of such data difficult and cumbersome. As a result, automatic text summarization is crucial for extracting relevant information, saving precious reading time. Although many transformer-based models excel in summarization, they are constrained by their input size, preventing them from processing texts longer than their context size. This study introduces three novel algorithms that allow any LLM to efficiently overcome its input size limitation, effectively utilizing its full potential without any architectural modifications. We test our algorithms on texts with more than 70,000 words, and our experiments show a significant increase in BERTScore with competitive ROUGE scores.
Anthology ID:
2024.icon-1.72
Volume:
Proceedings of the 21st International Conference on Natural Language Processing (ICON)
Month:
December
Year:
2024
Address:
AU-KBC Research Centre, Chennai, India
Editors:
Sobha Lalitha Devi, Karunesh Arora
Venue:
ICON
SIG:
Publisher:
NLP Association of India (NLPAI)
Note:
Pages:
607–615
Language:
URL:
https://aclanthology.org/2024.icon-1.72/
DOI:
Bibkey:
Cite (ACL):
Naman Chhibbar and Jugal Kalita. 2024. Automatic Summarization of Long Documents. In Proceedings of the 21st International Conference on Natural Language Processing (ICON), pages 607–615, AU-KBC Research Centre, Chennai, India. NLP Association of India (NLPAI).
Cite (Informal):
Automatic Summarization of Long Documents (Chhibbar & Kalita, ICON 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.icon-1.72.pdf