Narrative License and Model Sycophancy in LLM Summaries of Scientific Work

Calvin Isch, Grace Jennings


Abstract
Large language models (LLMs) are increasingly used to summarize academic work, yet model summaries can subtly exaggerate or mischaracterize findings. We examine how Narrative License (NL), rhetorical shifts that amplify claims beyond the underlying evidence, emerges in LLM summaries of scholarly articles. Using diverse prompting strategies across six leading models, we assess three dimensions of NL: causal overreach, rhetorical confidence, and sentiment (N = 100 peer-reviewed articles). Under basic summarization prompts, models frequently increase NL relative to academic abstracts; however, guardrail prompts can reduce these distortions. We further test how model "sycophancy" shapes NL, finding that stated stances and user personas produce predictable shifts in each element. These findings suggest that users and the benchmarks used to evaluate summarization should explicitly consider subtle rhetorical distortions and user alignment to ensure faithful scientific communication.
Anthology ID:
2026.acl-long.746
Volume:
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
16418–16432
Language:
URL:
https://aclanthology.org/2026.acl-long.746/
DOI:
10.18653/v1/2026.acl-long.746
Bibkey:
Cite (ACL):
Calvin Isch and Grace Jennings. 2026. Narrative License and Model Sycophancy in LLM Summaries of Scientific Work. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16418–16432, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
Narrative License and Model Sycophancy in LLM Summaries of Scientific Work (Isch & Jennings, ACL 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.acl-long.746.pdf
Checklist:
 2026.acl-long.746.checklist.pdf