Affective Computing Using an Emotion-Perception System for Interactive Experiences with Multi-Sensing Interfaces

Affective computing has emerged as a prominent technology in recent years, driven by the increasing demand for machines capable of understanding and interpreting emotions. The COVID-19 pandemic has intensified this need by reducing physical interactions and increasing reliance on digital communication, which often involves complex emotional exchanges. To enhance social interactions by identifying and addressing negative emotions, an interactive emotion-perception system based on multi-sensing interfaces and affective computing has been proposed. Integration of gesture, voice, and brainwave sensors is employed to provide users with a rich sensory experience and immersive environment. Design principles and system development processes are derived from the affinity diagram method and prototype development approach. The proposed system was constructed, and experiments were conducted with 60 participants. Interpersonal interaction scenarios were explored to understand emotional transmission mechanisms, aiming to enhance user satisfaction and positive interpersonal experiences. Data were collected through indirect observation and questionnaires, followed by evaluation and statistical analysis. Findings include: (1) The emotion-perception system enhances positive interpersonal interactions by identifying 36 distinct emotions, with overall user emotions being positive but exhibiting notable individual variations; (2) The system effectively stimulates positive emotions and strengthens interpersonal interactions, with affective computing technology fostering enjoyable emotional resonance and high-quality relationships; (3) A positive correlation was observed between semantic analysis and brainwave metrics of relaxation and concentration, showing increased concentration and enhanced positive emotional interactions during message transitions; and (4) The multi-sensing and affective computing technology provides a pleasant and relaxing emotional journey, with interactions shifting from enthusiasm to comfort, demonstrating the system’s capacity for a satisfying and enjoyable experience.