Shengyuan Wang
Author directoryOther people with similar names: Shengyuan Wang
Unverified author pages with similar names: Shengyuan Wang
2026
SurgAgents: A Multi-agent Framework for Embodied Cataract Surgical Intent Prediction
Shengyuan Wang | Yanzhe Ji | Yue Yao | Zhenyue Qin | Yang Liu
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Shengyuan Wang | Yanzhe Ji | Yue Yao | Zhenyue Qin | Yang Liu
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Predicting next surgical phase, viewed as a form of surgical procedural intent, with high accuracy is essential for robotic real-time surgical systems. Previous approaches struggle with short-term dynamics, occlusions, and phase persistence; end-to-end large vision-language models (LVLMs) can reason from visual inputs but often fail to exploit structured temporal observations. Thus, we introduce SurgAgents, a multi-agent framework for embodied surgical intent prediction. SurgAgents decomposes prediction into specialized agents: (1) a ResNet-based single-frame classifier agent extracts per-frame phase information; (2) a YOLO-based instrument specialist agent extracts per-frame instrument sets; (3) a history descriptor agent composes these facts into frame-level descriptions; (4) a history summarizer agent compresses long histories into a compact temporal narrative; (5) a phase predictor agent assembles the narrative and the current frame into a formatted LVLM prompt; (6) a retrieval agent with a confidence-gated Retrieval-augmented generation (RAG) module selectively retrieves similar case patterns when prediction confidence is low. On Cataract-1K, SurgAgents attains 46.54% macro-F1 with GPT-4o, outperforming the baseline, with systematic ablation studies showing that each agent materially contributes to performance. Latency measurements demonstrate feasibility for robotic real-time surgical support during surgery procedures.