Attribute or Abstain: Large Language Models as Long Document Assistants

Jan Buchmann, Xiao Liu, Iryna Gurevych


Abstract
LLMs can help humans working with long documents, but are known to hallucinate. *Attribution* can increase trust in LLM responses: The LLM provides evidence that supports its response, which enhances verifiability. Existing approaches to attribution have only been evaluated in RAG settings, where the initial retrieval confounds LLM performance. This is crucially different from the long document setting, where retrieval is not needed, but could help. Thus, a long document specific evaluation of attribution is missing. To fill this gap, we present LAB, a benchmark of 6 diverse long document tasks with attribution, and experiments with different approaches to attribution on 5 LLMs of different sizes. We find that *citation*, i.e. response generation and evidence extraction in one step, performs best for large and fine-tuned models, while additional retrieval can help for small, prompted models. We investigate whether the “Lost in the Middle” phenomenon exists for attribution, but do not find this. We also find that evidence quality can predict response quality on datasets with simple responses, but not so for complex responses, as models struggle with providing evidence for complex claims. We release code and data for further investigation. [Link](https://github.com/UKPLab/arxiv2024-attribute-or-abstain)
Anthology ID:
2024.emnlp-main.463
Volume:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8113–8140
Language:
URL:
https://aclanthology.org/2024.emnlp-main.463/
DOI:
10.18653/v1/2024.emnlp-main.463
Bibkey:
Cite (ACL):
Jan Buchmann, Xiao Liu, and Iryna Gurevych. 2024. Attribute or Abstain: Large Language Models as Long Document Assistants. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 8113–8140, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Attribute or Abstain: Large Language Models as Long Document Assistants (Buchmann et al., EMNLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.emnlp-main.463.pdf
Software:
 2024.emnlp-main.463.software.zip
Data:
 2024.emnlp-main.463.data.zip