Verifiable Generation with Subsentence-Level Fine-Grained Citations

Shuyang Cao, Lu Wang


Abstract
Verifiable generation requires large language models (LLMs) to cite source documents supporting their outputs, thereby improve output transparency and trustworthiness. Yet, previous work mainly targets the generation of sentence-level citations, lacking specificity about which parts of a sentence are backed by the cited sources. This work studies verifiable generation with subsentence-level fine-grained citations for more precise location of generated content supported by the cited sources. We first present a dataset, SCiFi, comprising 10K Wikipedia paragraphs with subsentence-level citations. Each paragraph is paired with a set of candidate source documents for citation and a query that triggers the generation of the paragraph content. On SCiFi, we evaluate the performance of state-of-the-art LLMs and strategies for processing long documents designed for these models. Our experiment results reveals key factors that could enhance the quality of citations, including the expansion of the source documents’ context accessible to the models and the implementation of specialized model tuning.
Anthology ID:
2024.findings-acl.920
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
15584–15596
Language:
URL:
https://aclanthology.org/2024.findings-acl.920
DOI:
Bibkey:
Cite (ACL):
Shuyang Cao and Lu Wang. 2024. Verifiable Generation with Subsentence-Level Fine-Grained Citations. In Findings of the Association for Computational Linguistics ACL 2024, pages 15584–15596, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
Verifiable Generation with Subsentence-Level Fine-Grained Citations (Cao & Wang, Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-acl.920.pdf