Deep Learning-Based Wearable Ear-EEG Emotion Recognition System With Superlets-Based Signal-to-Image Conversion Framework

This study presents a novel approach for affective computing through deep learning-assisted electroencephalography (EEG) analysis aimed at enhancing the detection, processing, interpretation, and emulation of human emotions. The proposed methodology introduces an Ear-EEG emotion recognition (EEER) system, leveraging a deep neural network and integrating the Internet of Things (IoT) capabilities. The key contributions of this study include 1) a comprehensive design framework encompassing both hardware and software aspects of the Ear-EEG system, as well as 2) a signals-to-three-channel image conversion method employing time-frequency super-resolution with superlets as inputs for deep learning models. Moreover, 3) a modified vision transformer (ViT) architecture is introduced, incorporating shifted patch tokenization (SPT) and locality self-attention (LSA) techniques, addressing challenges associated with limited and imbalanced small datasets, as well as the lack of locality inductive bias for emotion recognition. Additionally, 4) an IoT-assisted EEER platform is proposed, enabling remote monitoring and management. Experimental results indicate that the trained ViT model with SPT and LSA surpasses recent models in terms of performance on untrained datasets, achieving an average accuracy of 92.39%. The findings highlight the efficacy of the proposed EEER system in accurately detecting emotional states, encompassing positive and negative affective states. The integration of artificial intelligence and IoT-based healthcare platforms represents a significant advancement in the development of medical assistant tools with broad implications for future applications.