On smart production lines, workers flexibly follow standard operation procedures (SOPs) to meet changing demands while maintaining quality. To support this, this study developed a deep learning-based flexible operation guidance model. The model’s core consists of three procedures: (1) Object recognition: The model identifies worker motions like picking up, placing, and attaching objects. (2) Object state identification: The model identifies the state of an object (e.g., its appearance, position, and quantity) and uses a majority voting mechanism across multiple video frames to accurately determine the worker’s current motion based on a predefined SOP table by the proposed flexible SOP framework in the study. (3) Operator guidance: The model continuously assesses the worker’s actions against the SOP table. When it detects non-compliance or abnormalities, it provides real-time audio warnings for incorrect sequences, overtime, or foreign objects. This model was applied to a complex and flexible graphics processing units (GPUs) card assembly station, achieving an accuracy of 98.89%, precision of 99.86%, recall of 99.02%, and F1-score of 99.43% in overall for SOP motion detection, as well as a high mAP score of 99.85% for individual object detection, proving its feasibility and efficiency. The results also demonstrate a significant reduction in non-conformity by identifying incorrect sequences, timeouts, and foreign objects in real-time. By providing immediate audio guidance, the system ensures 100% adherence to flexible SOPs, directly improving first pass yield. The model’s high mAP across diverse operator profiles proves its ability to stabilize cycle time consistency. By modularizing motions, the proposed system effectively lowers the learning curve for novice operators, offering a scalable, automated quality assurance tool for high-precision manufacturing environments.
Wang et al. (2026) studied this question.