Multi-Review Fusion-in-Context

Aviv Slobodkin, Ori Shapira, Ran Levy, Ido Dagan


Abstract
Grounded text generation, encompassing tasks such as long-form question-answering and summarization, necessitates both content selection and content consolidation. Current end-to-end methods are difficult to control and interpret due to their opaqueness.Accordingly, recent works have proposed a modular approach, with separate components for each step. Specifically, we focus on the second subtask, of generating coherent text given pre-selected content in a multi-document setting. Concretely, we formalize Fusion-in-Context (FiC) as a standalone task, whose input consists of source texts with highlighted spans of targeted content. A model then needs to generate a coherent passage that includes all and only the target information.Our work includes the development of a curated dataset of 1000 instances in the reviews domain, alongside a novel evaluation framework for assessing the faithfulness and coverage of highlights, which strongly correlate to human judgment. Several baseline models exhibit promising outcomes and provide insightful analyses.This study lays the groundwork for further exploration of modular text generation in the multi-document setting, offering potential improvements in the quality and reliability of generated content. Our benchmark, FuseReviews, including the dataset, evaluation framework, and designated leaderboard, can be found at https://fusereviews.github.io/.
Anthology ID:
2024.findings-naacl.189
Volume:
Findings of the Association for Computational Linguistics: NAACL 2024
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Kevin Duh, Helena Gomez, Steven Bethard
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3003–3021
Language:
URL:
https://aclanthology.org/2024.findings-naacl.189
DOI:
Bibkey:
Cite (ACL):
Aviv Slobodkin, Ori Shapira, Ran Levy, and Ido Dagan. 2024. Multi-Review Fusion-in-Context. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3003–3021, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
Multi-Review Fusion-in-Context (Slobodkin et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-naacl.189.pdf