Qiuzhuang Sun

Author directory

2026

The manufacturing sector is increasingly adopting Multimodal Large Language Models (MLLMs) to move from low-level perception toward higher-level reasoning and decision support, yet current evaluations rarely reflect the fine-grained precision that real-world manufacturing demands. Progress is further hindered by data scarcity and a lack of fine-grained domain semantics in existing datasets. To bridge this gap, we introduce FORGE, a benchmark for three core quality-assurance tasks of discrete manufacturing: workpiece verification, structural surface inspection, and assembly verification. We first construct a high-quality multimodal dataset spanning real-world 2D images and 3D point clouds, annotated with fine-grained domain semantics such as standardized model numbers. We then evaluate 18 state-of-the-art MLLMs on these three tasks, revealing significant performance gaps. Counter to conventional understanding, our bottleneck analysis on five representative models indicates that visual grounding is not the primary limiting factor. Instead, insufficient domain-specific knowledge is the key bottleneck. Beyond evaluation, we show that our structured annotations can serve as an actionable training resource: supervised fine-tuning of a compact 3B-parameter model on our data yields up to 90.8% relative improvement in accuracy on held-out manufacturing scenarios, providing preliminary evidence for a practical pathway toward domain-adapted manufacturing MLLMs.