GenState-AI: State-Aware Dataset for Text-to-Video Retrieval on AI-Generated Videos

Minghan Li, Tongna Chen, Tianrui Lv, Yishuai Zhang, Suchao An, Guodong Zhou


Abstract
Existing text-to-video retrieval benchmarks often permit matching from static appearance, making it difficult to determine whether models capture state changes and final outcomes. We introduce GenState-AI, a benchmark comprising 956 AI-generated videos and 195 verified state-change queries across Toy, Cartoon, Home, and Manipulation domains. Its construction uses prompt-controlled variants that target either conflicting temporal/state outcomes or semantic substitutions. Following human verification, all valid videos contribute to subset-level retrieval pools, while complete query-specific triplets support diagnostic hard-negative ordering. We evaluate VCF-Lik, Qwen3-VL, and LanguageBind using both standard Recall@K and triplet-based consistency metrics. Qwen3-VL (8B) achieves the best macro-average R@1 and triplet consistency, while LanguageBind obtains the highest semantic discrimination accuracy. However, stronger retrieval does not consistently yield correct hard-negative ordering, and Qwen3-VL and LanguageBind remain weak on Manipulation triplets, where VCF-Lik performs substantially better. GenState-AI therefore provides a controlled diagnostic testbed for distinguishing global retrieval quality from fine-grained temporal and semantic grounding in AI-generated videos. Dataset: https://huggingface.co/datasets/Mh-Li/genstate-ai
Anthology ID:
2026.findings-aacl.124
Volume:
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2435–2448
Language:
URL:
https://aclanthology.org/2026.findings-aacl.124/
DOI:
Bibkey:
Cite (ACL):
Minghan Li, Tongna Chen, Tianrui Lv, Yishuai Zhang, Suchao An, and Guodong Zhou. 2026. GenState-AI: State-Aware Dataset for Text-to-Video Retrieval on AI-Generated Videos. In Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026, pages 2435–2448, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
GenState-AI: State-Aware Dataset for Text-to-Video Retrieval on AI-Generated Videos (Li et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-aacl.124.pdf