Reetam Biswas
Author directory2026
Ontology-Guided Procedural Knowledge Extraction and Validation Case Study on the Lockout-Tagout Documents
Cennet Oguz | Noon Pokaratsiri Goldstein | Jörg Steffen | Reetam Biswas | Günter Neumann
The Proceedings of Workshop on Document Intelligence and Understanding
Cennet Oguz | Noon Pokaratsiri Goldstein | Jörg Steffen | Reetam Biswas | Günter Neumann
The Proceedings of Workshop on Document Intelligence and Understanding
We present a case study developing an ontology-guided framework for extracting and validating procedural knowledge from safety-critical documents with limited annotations, specifically the Lockout-Tagout (LOTO) procedural documents. We build multimodal Procedural Knowledge Graphs (PKGs) grounded in the Procedural Knowledge Ontology (PKO) from text and diagrams using LLMs and VLMs. The extracted PKGs are evaluated on instruction- and graph-level metrics with validation against a domain expert curated ontology for action–entity relations, step order, and cross-modal alignment. Embedding-based metrics capture semantic alignment, while lexical metrics expose safety-critical mismatches. Our experiments on proprietary and public LOTO documents demonstrate the feasibility and limitations of multimodal procedural extraction and motivate ontology-guided validation.