A novel deep learning approach with feature injection technique for automated gait detection in parkinson’s disease using multi-modal data

Abstract Parkinson’s disease (PD) is a persistent, irreversible neurodegenerative condition, with freezing of gait (FOG) being among its most incapacitating symptoms. PD patients with FOG encounter abrupt and unforeseen difficulties in initiating or maintaining movement, potentially resulting in unexpected falls and injuries. A sensor-driven FOG detection system can facilitate continuous and objective tracking of PD patients experiencing FOG and provide on-the-spot cueing support. Many recent studies have employed advanced deep learning methods to detect FOG by analyzing data from only inertial measurement unit (IMU) sensors. In our study, we have utilized a multimodal dataset that includes a gait accelerometer (ACC), electromyogram (EMG), and electroencephalogram (EEG) for FOG detection. For robust FOG detection task, we have proposed a novel deep learning-based model called Self-FOGNet, which integrates the self-organized operational neuron (Self-ONN) layer to enhance the learning of intricate patterns within the data by introducing non-homogeneity in neural networks. The robust model also promotes feature reuse and efficiently utilizes the hierarchical feature space across multiple layers, enabling effective classification. Further, we extracted several wavelet features and injected them at the last layer of the model as time domain and frequency domain features. We have used 3 different models for 3 data modalities and used an ensemble learning approach to combine the outputs of each model to get the final improved prediction. As a result of our proposed method, we achieved 98.74% accuracy and 98.68% specificity in the FOG detection task with an ensemble learning approach that outperformed the state-of-the-art approaches.