Shengyuan Wang

Author directory

Other people with similar names: Shengyuan Wang

Unverified author pages with similar names: Shengyuan Wang


2026

Predicting next surgical phase, viewed as a form of surgical procedural intent, with high accuracy is essential for robotic real-time surgical systems. Previous approaches struggle with short-term dynamics, occlusions, and phase persistence; end-to-end large vision-language models (LVLMs) can reason from visual inputs but often fail to exploit structured temporal observations. Thus, we introduce SurgAgents, a multi-agent framework for embodied surgical intent prediction. SurgAgents decomposes prediction into specialized agents: (1) a ResNet-based single-frame classifier agent extracts per-frame phase information; (2) a YOLO-based instrument specialist agent extracts per-frame instrument sets; (3) a history descriptor agent composes these facts into frame-level descriptions; (4) a history summarizer agent compresses long histories into a compact temporal narrative; (5) a phase predictor agent assembles the narrative and the current frame into a formatted LVLM prompt; (6) a retrieval agent with a confidence-gated Retrieval-augmented generation (RAG) module selectively retrieves similar case patterns when prediction confidence is low. On Cataract-1K, SurgAgents attains 46.54% macro-F1 with GPT-4o, outperforming the baseline, with systematic ablation studies showing that each agent materially contributes to performance. Latency measurements demonstrate feasibility for robotic real-time surgical support during surgery procedures.