Hongzhe Liu
Author directory2026
Testing Video MLLM Sensitivity to Hand-Localized Visual Detail in Chinese Sign Recognition: A Matched Visual Intervention Study
Bowen Ye | Dengfeng Yao | Siyuan Jing | Peng Jin | Hongzhe Liu | Cheng Xu
Proceedings of the Second Workshop on Sign Language Processing
Bowen Ye | Dengfeng Yao | Siyuan Jing | Peng Jin | Hongzhe Liu | Cheng Xu
Proceedings of the Second Workshop on Sign Language Processing
Recent benchmarks show that multimodal large language models (MLLMs) can answer questions about sign-language video, but accuracy alone does not reveal which visual cues support those answers. We test behavioral sensitivity to hand-localized visual detail using 150 front-view isolated signs from NationalCSL-DP. For each item, a hand-target condition (T) blurs tracked hand regions and a same-arm control (C) applies the same Gaussian operator, nominal σ, and translated mask geometry to nearby arm regions; the underlying regional content is not information-matched. The same four-choice question is used across conditions and four video MLLMs. All 1,800 final Primary responses are uniquely scorable, with 1,200 T/C responses entering the primary contrast. Mean accuracy is 38.17% on original videos (O), 33.50% on T, and 36.67% on C. The pre-specified C–T contrast is +3.17 percentage points (95% bootstrap CI [−0.17, 6.50]; item-cluster sign-flip p = 0.08172). The four model-level point estimates are descriptively positive, but two models are floor-limited. The evidence is therefore directional rather than confirmatory. The contribution is a matched-control diagnostic that complements aggregate benchmark accuracy while making its residual anatomical-content mismatch explicit.