Jie-Jing Shao

Author directory

2026

As natural language becomes a key interface for controlling text-to-image generation, users specify not only which visual concepts should appear but also how entities are constrained by attributes, relations, quantifiers, and cross-entity dependencies. Reliable controllable generation therefore requires generalization over semantic structure, not merely over novel combinations of concepts. Existing benchmarks mainly evaluate conceptual compositional generalization, while leaving the internal semantic structure of instructions underexplored. We study structural compositional generalization: whether text-to-image models can generalize across increasingly coupled semantic dependencies. We introduce FormalImG, a first-order-logic-based benchmark with 4,000 instructions across three complexity levels in two scenarios, together with computable structural compositional complexity and ε-structural compositional generalizability. Experiments on advanced text-to-image models show consistent degradation as structural complexity increases; even the strongest evaluated model generalizes only up to G0.7 = 6 in Natural and G0.7 = 4 in Knolling. Further analysis suggests that the main bottleneck lies in language-to-vision instantiation and can be partially mitigated by explicit intermediate layouts. The benchmark and code are available at https://github.com/yhj137/FormalImG.