FineRAG: Fine-grained Retrieval-Augmented Text-to-Image Generation

Huaying Yuan, Ziliang Zhao, Shuting Wang, Shitao Xiao, Minheng Ni, Zheng Liu, Zhicheng Dou


Abstract
Recent advancements in text-to-image generation, notably the series of Stable Diffusion methods, have enabled the production of diverse, high-quality photo-realistic images. Nevertheless, these techniques still exhibit limitations in terms of knowledge access. Retrieval-augmented image generation is a straightforward way to tackle this problem. Current studies primarily utilize coarse-grained retrievers, employing initial prompts as search queries for knowledge retrieval. This approach, however, is ineffective in accessing valuable knowledge in long-tail text-to-image generation scenarios. To alleviate this problem, we introduce FineRAG, a fine-grained model that systematically breaks down the retrieval-augmented image generation task into four critical stages: query decomposition, candidate selection, retrieval-augmented diffusion, and self-reflection. Experimental results on both general and long-tailed benchmarks show that our proposed method significantly reduces the noise associated with retrieval-augmented image generation and performs better in complex, open-world scenarios.
Anthology ID:
2025.coling-main.741
Volume:
Proceedings of the 31st International Conference on Computational Linguistics
Month:
January
Year:
2025
Address:
Abu Dhabi, UAE
Editors:
Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, Steven Schockaert
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
11196–11205
Language:
URL:
https://aclanthology.org/2025.coling-main.741/
DOI:
Bibkey:
Cite (ACL):
Huaying Yuan, Ziliang Zhao, Shuting Wang, Shitao Xiao, Minheng Ni, Zheng Liu, and Zhicheng Dou. 2025. FineRAG: Fine-grained Retrieval-Augmented Text-to-Image Generation. In Proceedings of the 31st International Conference on Computational Linguistics, pages 11196–11205, Abu Dhabi, UAE. Association for Computational Linguistics.
Cite (Informal):
FineRAG: Fine-grained Retrieval-Augmented Text-to-Image Generation (Yuan et al., COLING 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.coling-main.741.pdf