Jiaming Han
Author directory2026
ConceptGuard: Proactive Safety in Text-and-Image-to-Video Generation through Multimodal Risk Detection
Ruize Ma | Minghong Cai | Yilei Jiang | Jiaming Han | Yi Feng | Yingshui Tan | Bo Zhang | Xiangyu Yue
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Ruize Ma | Minghong Cai | Yilei Jiang | Jiaming Han | Yi Feng | Yingshui Tan | Bo Zhang | Xiangyu Yue
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Recent progress in video generative models has enabled the creation of high-quality videos from multimodal prompts that combine text and images. While these systems offer greater controllability, they also introduce new safety risks, as harmful content can arise from individual modalities or their interaction. Existing safety methods are often limited to text-only inputs, require the user to manually specify the risk category in advance, or only audit content after generation, making them unable to proactively address compositional multimodal risks. We present ConceptGuard, a safeguard framework that proactively detects and mitigates unsafe semantics in multimodal video generation before harmful content is produced. ConceptGuard operates in two stages. First, a contrastive detection module fuses image and text inputs and computes their similarity against the risk concept set , automatically retrieving the top-k concepts most relevant to the input without requiring any prior assumption about the risk category. Second, a semantic suppression mechanism uses these identified concepts to remove unsafe semantics from both the text and the visual input during early generation steps. To support training and evaluation, we introduce ConceptRisk, a large-scale dataset for multimodal risk detection, and T2VSafetyBench-TI2V, the first benchmark for evaluating safety in Text-and-Image-to-Video generation. Experiments on both benchmarks show that ConceptGuard outperforms existing baselines in both risk detection and safe video generation.
ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agents
Yilei Jiang | Yaozhi Zheng | Yuxuan Wan | Jiaming Han | Qunzhong Wang | Michael R. Lyu | Xiangyu Yue
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yilei Jiang | Yaozhi Zheng | Yuxuan Wan | Jiaming Han | Qunzhong Wang | Michael R. Lyu | Xiangyu Yue
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Automating the transformation of user interface (UI) designs into front-end code holds significant promise for accelerating software development and democratizing design workflows. While multimodal large language models (MLLMs) can translate images to code, they often fail on complex UIs, struggling to unify visual perception, layout planning, and code synthesis within a single monolithic model, which leads to frequent perception and planning errors. To address this, we propose ScreenCoder, a modular multi-agent framework that decomposes the task into three interpretable stages: grounding, planning, and generation. By assigning these distinct responsibilities to specialized agents, our framework achieves significantly higher robustness and fidelity than end-to-end approaches. Furthermore, ScreenCoder serves as a scalable data engine, enabling us to generate high-quality image-code pairs. We use this data to fine-tune open-source MLLM via a dual-stage pipeline of supervised fine-tuning and reinforcement learning, demonstrating substantial gains in its UI generation capabilities. Extensive experiments demonstrate that our approach achieves state-of-the-art performance in layout accuracy, structural coherence, and code correctness.
Probing Audio-Visual Reasoning in Multimodal Language Models through the Lens of Audio
Kaixiong Gong | Kaituo Feng | Bohao Li | Yibing Wang | Mofan Cheng | Shijia Yang | Jiaming Han | Benyou Wang | Yutong Bai | Zhuoran Yang | Xiangyu Yue
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Kaixiong Gong | Kaituo Feng | Bohao Li | Yibing Wang | Mofan Cheng | Shijia Yang | Jiaming Han | Benyou Wang | Yutong Bai | Zhuoran Yang | Xiangyu Yue
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Recent multimodal large language models (MLLMs), such as GPT-4o, Gemini 1.5/2.5 Pro, and Reka Core, have advanced audio-visual reasoning capabilities, achieving strong performance in tasks like cross-modal understanding and generation. However, our DeafTest uncovers unanticipated failures: most of the state-of-the-art MLLMs struggle with very simple audio tasks, such as distinguishing louder sounds or sound counting. This raises a fundamental question—does a deficiency in low-level audio perception constrain higher-level audio-visual reasoning? To address this, we introduce AV-Odyssey Bench—a comprehensive benchmark of 4,555 meticulously designed problems that integrate text, audio, and visual modalities. Each task requires models to unify cross-modal reasoning, leveraging synchronized audio-visual cues to infer solutions. By structuring questions as multiple-choice, we ensure objective, reproducible evaluations without reliance on subjective human or LLM-based judgments. Through comprehensive benchmarking of closed-source and open-source models, we showcase: (i) current MLLMs lack robust audio-visual integration ability and (ii) performance on DeafTest (Pearson’s r = 0.945) strongly correlates with AV-Odyssey accuracy. These findings challenge assumptions about models’ multimodal proficiency and highlight fundamental audio perception as a reasoning bottleneck. We believe that our results provide concrete guidance for future dataset design, alignment strategies, and architectures.