Extended working hours and repetitive industrial tasks significantly contribute to worker fatigue and stress, increasing the likelihood of workplace accidents and reducing productivity. This paper presents an advanced Real-Time Worker Fatigue and Stress Detection System that leverages deep learning, computer vision, and multimodal analytics for proactive risk monitoring. The proposed system integrates a FastAPI-based backend with a React frontend to perform real-time analysis of video and audio inputs. Unlike traditional systems, it employs YOLOv8 for high-speed multi-face detection, with MediaPipe as a fallback mechanism to ensure robustness. Facial regions are enhanced using CLAHE and Retinex preprocessing to handle illumination variability. Emotion classification is performed using a CNN trained on the FER-2013 dataset and refined using a Temporal Transformer model to capture sequential dependencies. The system further incorporates Deep SORT for multi-person tracking and DeepFace for identity recognition, enabling personalized monitoring. A multimodal pipeline integrates voice-based stress detection using acoustic features. A dynamic risk scoring engine computes worker risk levels using weighted contributions from fatigue, stress, and session duration, with worker-specific calibration. MongoDB is used for real-time and historical analytics. The system processes data in memory to ensure privacy. Overall, the solution provides a scalable, intelligent, and non-intrusive approach to workplace safety.
Kesavan et al. (Sun,) studied this question.