Niantong Li

Author directory

2026

Existing Text-to-Image (T2I) benchmarks remain anchored in foundational criteria such as image quality and text-image alignment, failing to capture the nuanced capabilities that professional creators demand in real-world workflows. Meanwhile, evaluation pipelines that rely on an untrained multimodal model as a judge diverge from human standards. We introduce Qwen-Image-Bench, a creator-centric benchmark that extends evaluation with two application-driven dimensions (Real-world Fidelity and Creative Generation), alongside the conventional pillars of Quality, Aesthetics, and Alignment. The benchmark organizes these into a three-level taxonomy (5 pillars, 23 sub-capabilities, 56 verifiable rubrics) and provides 1000 stratified bilingual prompts. We train a unified Judge Model (Q-Judger) on Qwen3.6-27B, supervised by 80 professional annotators under blind labeling and triple-review protocols, that scores every image across all 56 facets with fine-grained, rubric-grounded diagnostics. Empirically, Qwen-Image-Bench reliably distinguishes leading T2I models, achieving the greatest separation on the application-driven dimensions where existing benchmarks provide limited insight.