Jingjing Chen
Author directory2026
RoboTrustBench: Benchmarking the Trustworthiness of Video World Models for Robotic Manipulation
Huiqiong Li | Jiayu Wang | Zhiting Mei | Anirudha Majumdar | Jingjing Chen | Bin Zhu
Findings of the Association for Computational Linguistics: EMNLP 2026
Huiqiong Li | Jiayu Wang | Zhiting Mei | Anirudha Majumdar | Jingjing Chen | Bin Zhu
Findings of the Association for Computational Linguistics: EMNLP 2026
Video world models are increasingly used in robotic manipulation, yet existing benchmarks mostly evaluate them under valid, feasible, and safe instructions. We introduce RoboTrustBench, a benchmark for evaluating the trustworthiness of video world models under four scenarios: Normal, Constraint-Sensitive, Counterfactual, and Adversarial. Built from real-world DROID episodes, RoboTrustBench contains 1,207 expert-validated instruction–image pairs and a six-dimensional evaluation protocol with 13 fine-grained criteria. Evaluating seven representative video world models with human and MLLM assessment, we find that current models often generate visually coherent videos, but struggle with constraint reasoning, counterfactual grounding, physical interaction, and unsafe-instruction suppression. These results show that visual quality and surface-level instruction following are insufficient for trustworthy robotic video world modeling.
Spatiotemporal Sycophancy: Negation-Based Gaslighting in Video Large Language Models
Ziyao Tang | Pengkun Jiao | Bin Zhu | Huiyan Qi | Jingjing Chen | Yu-Gang Jiang
Findings of the Association for Computational Linguistics: ACL 2026
Ziyao Tang | Pengkun Jiao | Bin Zhu | Huiyan Qi | Jingjing Chen | Yu-Gang Jiang
Findings of the Association for Computational Linguistics: ACL 2026
Video Large Language Models (Vid-LLMs) have demonstrated remarkable performance in video understanding tasks, yet their robustness under conversational interaction remains largely underexplored. In this paper, we identify spatiotemporal sycophancy, a failure mode in which Vid-LLMs retract initially correct, visually grounded judgments and conform to misleading user feedback under negation-based gaslighting. Rather than merely changing their answers, the models often fabricate unsupported temporal or spatial explanations to justify incorrect revisions. To systematically investigate this phenomenon, we propose a negation-based gaslighting evaluation framework and introduce GasVideo-1000, a curated benchmark designed to probe spatiotemporal sycophancy with clear visual grounding and temporal reasoning requirements. We evaluate a broad range of state-of-the-art open-source and proprietary Vid-LLMs across diverse video understanding tasks. Extensive experiments reveal that vulnerability to negation-based gaslighting is pervasive and severe, even among models with strong baseline performance. While prompt-level grounding constraints can partially mitigate this behavior, they do not reliably prevent hallucinated justifications or belief reversal. Our results indicate that current Vid-LLMs lack robust mechanisms for maintaining grounded spatiotemporal beliefs under adversarial conversational feedback.
OffSeeker: Online Reinforcement Learning Is Not All You Need for Deep Research Agents
Yuhang Zhou | Kai Zheng | Qiguang Chen | Mengkang Hu | Qingfeng Sun | Can Xu | Jingjing Chen
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yuhang Zhou | Kai Zheng | Qiguang Chen | Mengkang Hu | Qingfeng Sun | Can Xu | Jingjing Chen
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Deep research agents have shown remarkable potential in handling long-horizon tasks. However, state-of-the-art performance typically relies on online reinforcement learning (RL), which is financially expensive due to extensive API calls. While offline training offers a more efficient alternative, its progress is hindered by the scarcity of high-quality research trajectories. In this paper, we demonstrate that expensive online reinforcement learning is not all you need to build powerful research agents. To bridge this gap, we introduce a fully open-source suite designed for effective offline training. Our core contributions include DeepForge, a ready-to-use task synthesis framework that generates large-scale research queries without heavy preprocessing; and a curated collection of 66k QA pairs, 33k SFT trajectories, and 21k DPO pairs. Leveraging these resources, we train OffSeeker, a model developed entirely offline. Extensive evaluations across six benchmarks show that OffSeeker not only leads among similar-sized agents but also remains competitive with 30B-parameter systems trained via heavy online RL.
OSCBench: Benchmarking Object State Change in Text-to-Video Generation
Xianjing Han | Bin Zhu | Shiqi Hu | Franklin Mingzhe Li | Patrick Carrington | Roger Zimmermann | Jingjing Chen
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Xianjing Han | Bin Zhu | Shiqi Hu | Franklin Mingzhe Li | Patrick Carrington | Roger Zimmermann | Jingjing Chen
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Text-to-video (T2V) generation models have made rapid progress in producing visually high-quality and temporally coherent videos. However, existing benchmarks primarily focus on perceptual quality, text–video alignment, or physical plausibility, leaving a critical aspect of action understanding largely unexplored: object state change (OSC) explicitly specified in the text prompt. OSC refers to the transformation of an object’s state induced by an action, such as peeling a potato or slicing a lemon. In this paper, we introduce OSCBench, a benchmark specifically designed to assess OSC performance in T2V models. OSCBench is constructed from instructional cooking data and systematically organizes action–object interactions into regular, novel, and compositional scenarios to probe both in-distribution performance and generalization. We evaluate six representative open-source and proprietary T2V models using both human user study and multimodal large language model (MLLM)–based automatic evaluation. Our results show that, despite strong performance on semantic and scene alignment, current T2V models consistently struggle with accurate and temporally consistent object state changes, especially in novel and compositional settings. These findings position OSC as a key bottleneck in text-to-video generation and establish OSCBench as a diagnostic benchmark for advancing state-aware video generation models. Project page: https://hanxjing.github.io/OSCBench.
2025
DLPO: Towards a Robust, Efficient, and Generalizable Prompt Optimization Framework from a Deep-Learning Perspective
Dengyun Peng | Yuhang Zhou | Qiguang Chen | JinHao Liu | Jingjing Chen | Libo Qin | Wanxiang Che
Findings of the Association for Computational Linguistics: EMNLP 2025
Dengyun Peng | Yuhang Zhou | Qiguang Chen | JinHao Liu | Jingjing Chen | Libo Qin | Wanxiang Che
Findings of the Association for Computational Linguistics: EMNLP 2025
Large Language Models (LLMs) have achieved remarkable success across diverse tasks, largely driven by well-designed prompts. However, crafting and selecting such prompts often requires considerable human effort, significantly limiting its scalability. To mitigate this, recent studies have explored automated prompt optimization as a promising solution. Despite these efforts, existing methods still face critical challenges in robustness, efficiency, and generalization. To systematically address these challenges, we first conduct an empirical analysis to identify the limitations of current reflection-based prompt optimization paradigm. Building on these insights, we propose 7 innovative approaches inspired by traditional deep learning paradigms for prompt optimization (DLPO), seamlessly integrating these concepts into text-based gradient optimization. Through these advancements, we progressively tackle the aforementioned challenges and validate our methods through extensive experimentation. We hope our study not only provides valuable guidance for future research but also offers a comprehensive understanding of the challenges and potential solutions in prompt optimization.
2023
Unifying Cross-Lingual and Cross-Modal Modeling Towards Weakly Supervised Multilingual Vision-Language Pre-training
Zejun Li | Zhihao Fan | Jingjing Chen | Qi Zhang | Xuanjing Huang | Zhongyu Wei
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Zejun Li | Zhihao Fan | Jingjing Chen | Qi Zhang | Xuanjing Huang | Zhongyu Wei
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Multilingual Vision-Language Pre-training (VLP) is a promising but challenging topic due to the lack of large-scale multilingual image-text pairs. Existing works address the problem by translating English data into other languages, which is intuitive and the generated data is usually limited in form and scale. In this paper, we explore a more practical and scalable setting: weakly supervised multilingual VLP with only English image-text pairs and multilingual text corpora. We argue that the universal multilingual representation learned from texts allows the cross-modal interaction learned in English to be transferable to other languages. To this end, we propose a framework to effectively unify cross-lingual and cross-modal pre-training. For unified modeling on different data, we design an architecture with flexible modules to learn different interactions. Moreover, two unified tasks are introduced to efficiently guide the unified cross-lingual cross-modal learning. Extensive experiments demonstrate that our pre-trained model learns universal multilingual multimodal representations, allowing effective cross-lingual transfer on multimodal tasks. Code and models are available at https://github.com/FudanDISC/weakly-supervised-mVLP.
Search
Fix author
Co-authors
- Bin Zhu 3
- Qiguang Chen (陈麒光) 2
- Yuhang Zhou 2
- Patrick Carrington 1
- Wanxiang Che (车万翔) 1
- Zhihao Fan 1
- Xianjing Han 1
- Mengkang Hu 1
- Shiqi Hu 1
- Xuan-Jing Huang (黄萱菁) 1
- Yu-Gang Jiang 1
- Pengkun Jiao 1
- Franklin Mingzhe Li 1
- Huiqiong Li 1
- Zejun Li (李泽君) 1
- Jinhao Liu 1
- Anirudha Majumdar 1
- Zhiting Mei 1
- Dengyun Peng 1
- Huiyan Qi 1
- Libo Qin 1
- Qingfeng Sun 1
- Ziyao Tang 1
- Jiayu Wang 1
- Zhongyu Wei (魏忠钰) 1
- Can Xu 1
- Qi Zhang 1
- Kai Zheng 1
- Roger Zimmermann 1