Siyuan Jing
Author directory2026
Testing Video MLLM Sensitivity to Hand-Localized Visual Detail in Chinese Sign Recognition: A Matched Visual Intervention Study
Bowen Ye | Dengfeng Yao | Siyuan Jing | Peng Jin | Hongzhe Liu | Cheng Xu
Proceedings of the Second Workshop on Sign Language Processing
Bowen Ye | Dengfeng Yao | Siyuan Jing | Peng Jin | Hongzhe Liu | Cheng Xu
Proceedings of the Second Workshop on Sign Language Processing
Recent benchmarks show that multimodal large language models (MLLMs) can answer questions about sign-language video, but accuracy alone does not reveal which visual cues support those answers. We test behavioral sensitivity to hand-localized visual detail using 150 front-view isolated signs from NationalCSL-DP. For each item, a hand-target condition (T) blurs tracked hand regions and a same-arm control (C) applies the same Gaussian operator, nominal σ, and translated mask geometry to nearby arm regions; the underlying regional content is not information-matched. The same four-choice question is used across conditions and four video MLLMs. All 1,800 final Primary responses are uniquely scorable, with 1,200 T/C responses entering the primary contrast. Mean accuracy is 38.17% on original videos (O), 33.50% on T, and 36.67% on C. The pre-specified C–T contrast is +3.17 percentage points (95% bootstrap CI [−0.17, 6.50]; item-cluster sign-flip p = 0.08172). The four model-level point estimates are descriptively positive, but two models are floor-limited. The evidence is therefore directional rather than confirmatory. The contribution is a matched-control diagnostic that complements aggregate benchmark accuracy while making its residual anatomical-content mismatch explicit.
2025
面向信息处理的中国手语音系学标注加工规范
Yuan Zhao | Peng Jin | Siyuan Jing | Dengfeng Yao | Ziwen Meng
Proceedings of the 24th China National Conference on Computational Linguistics (CCL 2025)
Yuan Zhao | Peng Jin | Siyuan Jing | Dengfeng Yao | Ziwen Meng
Proceedings of the 24th China National Conference on Computational Linguistics (CCL 2025)
“随着对手语进行大规模数据化处理的需求日益增强,手语的音系学标注及规范化工作愈发迫切。然而,手语作为一种视觉-空间语言,不同于有声语言,其多信道(手形、位置、手掌朝向、运动方式以及面部表情、躯干动作等非手动特征)信息的复杂性与缺乏统一标注规范,一直制约着手语语料库构建与自动分析技术的发展。针对这一问题,本研究在手语音系学理论的指导下,提出了一套面向中国手语音系学标注加工的系统化规范。该规范由原则和细则两部分构成:原则部分明确标注对象的粒度、标注单位的界定与分层方式;细则部分则给出多信道特征的具体标注实例与操作指南。该规范的实施为中国手语多信道特征的系统标注提供基础支撑,将有助于推动手语识别、翻译、生成以及教学平台的深入发展,加速中国手语信息处理标准化与规范化的进程。”