R-BASS : Relevance-aided Block-wise Adaptation for Speech Summarization

Roshan Sharma, Ruchira Sharma, Hira Dhamyal, Rita Singh, Bhiksha Raj


Abstract
End-to-end speech summarization on long recordings is challenging because of the high computational cost. Block-wise Adaptation for Speech Summarization (BASS) summarizes arbitrarily long sequences by sequentially processing abutting chunks of audio. Despite the benefits of BASS, it has higher compute time due to sequential processing of all blocks, regardless of whether they are relevant to the final summary. In this paper, we propose R-BASS, a new relevance-aware block-wise adaptation method. First, we introduce two approaches to automatically estimate block relevance based on lexical and semantic similarity between the block-level transcript and the summary. Experiments on the How2 dataset show that using ground truth relevance during inference improves efficiency by 63.9 % by dropping irrelevant blocks. Finally, we incorporate relevance scores into training using a novel relevance loss and relevance predictor, and the proposed R-BASS model makes it possible to drop 86.3 % of the blocks while retaining comparable performance, resulting in a 2.2x speedup over BASS.
Anthology ID:
2024.findings-naacl.54
Volume:
Findings of the Association for Computational Linguistics: NAACL 2024
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Kevin Duh, Helena Gomez, Steven Bethard
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
848–857
Language:
URL:
https://aclanthology.org/2024.findings-naacl.54
DOI:
Bibkey:
Cite (ACL):
Roshan Sharma, Ruchira Sharma, Hira Dhamyal, Rita Singh, and Bhiksha Raj. 2024. R-BASS : Relevance-aided Block-wise Adaptation for Speech Summarization. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 848–857, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
R-BASS : Relevance-aided Block-wise Adaptation for Speech Summarization (Sharma et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-naacl.54.pdf
Copyright:
 2024.findings-naacl.54.copyright.pdf