Another Turn, Better Output? A Turn-Wise Analysis of Iterative LLM Prompting

Shashidhar Reddy Javaji, Bhavul Gauri, Zining Zhu


Abstract
Large language models are increasingly used through multi-turn refinement loops, but most evidence for iterative improvement comes from settings with informative feedback, external tools, full conversation history, or structured self-critique. Less is known about a configuration, uninformative memoryless refinement, where a model repeatedly revises only its previous output under vague instructions such as “improve it”. In this paper, we establish a framework to analyze when this uninformative memoryless refinement configuration improves LLM responses across ideation, coding, and mathematical reasoning. We evaluate seven LLMs on a 450-task pool under vague and domain-specific feedback. We score every turn and measure both task outcomes and trajectory dynamics, including semantic drift, volatility, lexical novelty, and output growth. This trajectory view reveals a consistent discovery–retention gap: models often produce better intermediate outputs before overwriting them. The effect is most severe in coding, where correctness falls from 40.7% at T1 to 2.5% at T12, while best@12 exceeds final@12 by 42.5 percentage points. Math shows model- and prompt-dependent behavior, with GPT-OSS-20B improving under elaboration but several models drifting downward. Ideation benefits from explicit steering, but vague feedback produces rhetorical inflation and feasibility loss.Our analysis stresses the importance of turn-wise, trajectory-aware evaluation of uninformative memoryless refinement and shows why verified intermediate outputs should be retained rather than automatically replaced by later revisions.
Anthology ID:
2026.emnlp-main.2337
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
53085–53110
Language:
URL:
https://aclanthology.org/2026.emnlp-main.2337/
DOI:
Bibkey:
Cite (ACL):
Shashidhar Reddy Javaji, Bhavul Gauri, and Zining Zhu. 2026. Another Turn, Better Output? A Turn-Wise Analysis of Iterative LLM Prompting. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 53085–53110, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Another Turn, Better Output? A Turn-Wise Analysis of Iterative LLM Prompting (Javaji et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.2337.pdf