Xiao-Wen Yang
Author directory2026
FormalImG: Evaluating Structural Compositional Generalization for T2I Models
Hong-Jie You | Jie-Jing Shao | Xiao-Wen Yang | Zhi-Fan Wu | Lin-Han Jia | Lan-Zhe Guo | Yu-Feng Li
Findings of the Association for Computational Linguistics: EMNLP 2026
Hong-Jie You | Jie-Jing Shao | Xiao-Wen Yang | Zhi-Fan Wu | Lin-Han Jia | Lan-Zhe Guo | Yu-Feng Li
Findings of the Association for Computational Linguistics: EMNLP 2026
As natural language becomes a key interface for controlling text-to-image generation, users specify not only which visual concepts should appear but also how entities are constrained by attributes, relations, quantifiers, and cross-entity dependencies. Reliable controllable generation therefore requires generalization over semantic structure, not merely over novel combinations of concepts. Existing benchmarks mainly evaluate conceptual compositional generalization, while leaving the internal semantic structure of instructions underexplored. We study structural compositional generalization: whether text-to-image models can generalize across increasingly coupled semantic dependencies. We introduce FormalImG, a first-order-logic-based benchmark with 4,000 instructions across three complexity levels in two scenarios, together with computable structural compositional complexity and ε-structural compositional generalizability. Experiments on advanced text-to-image models show consistent degradation as structural complexity increases; even the strongest evaluated model generalizes only up to G0.7 = 6 in Natural and G0.7 = 4 in Knolling. Further analysis suggests that the main bottleneck lies in language-to-vision instantiation and can be partially mitigated by explicit intermediate layouts. The benchmark and code are available at https://github.com/yhj137/FormalImG.
2025
AutoEvolve: Automatically Evolving Queries for Applicable and Scalable Retrieval-Augmented Generation Benchmarking
Ding-Chu Zhang | Xiaowen Zhang | Yue Fei | Renjun Hu | Xiao-Wen Yang | Zhi Zhou | Baixuan Li | Yu-Feng Li | Xing Shi | Wei Lin
Findings of the Association for Computational Linguistics: EMNLP 2025
Ding-Chu Zhang | Xiaowen Zhang | Yue Fei | Renjun Hu | Xiao-Wen Yang | Zhi Zhou | Baixuan Li | Yu-Feng Li | Xing Shi | Wei Lin
Findings of the Association for Computational Linguistics: EMNLP 2025
Retrieval-augmented generation (RAG) enables large language models (LLMs) to address queries beyond their internal knowledge by integrating domain knowledge in specialized corpus, which necessitates the generation of benchmarks on specific corpus to evaluate RAG systems. However, existing automated generation methods exhibit Weak Applicability and Weak Scalability. Weak Applicability refers to the reliance on metadata from specific corpora for query generation, constraining applicability to other corpora. Weak Scalability is characterized by fixed query content after generation, unable to dynamically increase difficulty, limiting scalability of the query. To overcome these issues, we propose AutoEvolve, an applicable approach for dynamically evolving queries to construct scalable RAG benchmarks. Our approach is grounded in three key innovations: (i) a corpus-agnostic method for constructing the universal entity-document graph; (ii) a suite of evolution operations designed to dynamically update queries; and (iii) a difficulty-guided metric that directs query evolution process. Through experiments on three generated benchmarks, we demonstrate that AutoEvolve evolves queries that are significantly more challenging, paving the way for more applicable and scalable RAG evaluations.