Advances in Explainable AI for Affective Computing and Activity Recognition in Elderly

eng The global population is aging at an unprecedented rate, posing significant challenges in healthcare and social services—particularly due to the high incidence of falls and other age-related risks. At the same time, deep learning has revolutionized computer vision by enabling state-of-the-art performance in tasks such as facial expression recognition (FER) and human activity recognition (HAR), which are crucial for monitoring both the physical and emotional well-being of older adults. However, existing approaches often fail to deliver robust performance on specific populations, and the "black-box" nature of deep learning models limits their transparency and trustworthiness—an issue that is especially critical in high-stakes domains like healthcare, where explainable artificial intelligence (XAI) is essential. This thesis aims to improve the quality of life for the elderly by advancing facial expression recognition and human activity recognition through deep learning techniques, while placing special emphasis on the interpretability of these models via explainable artificial intelligence tools. To achieve this goal, we began by conducting two systematic literature reviews that mapped current deep learning approaches for FER and HAR and identified critical gaps—such as the underrepresentation of older adults in existing datasets and the limited use of XAI tools. In response to these challenges, we pursued several interrelated studies. To enhance FER in older adults, we first compiled a diverse collection of existing datasets and evaluated them using novel similarity metrics, and then developed innovative methods to address age biases by leveraging XAI tools to explore differences between age groups. For HAR, recognizing the scarcity of XAI techniques for video data, we modified an agnostic perturbation-based XAI method to derive separate spatial and temporal explanations and designed a framework that adapts common image and tabular-based XAI techniques to the video domain. Finally, to better understand human trust in AI systems, we investigated the similarities between human and machine performance in FER, examined how various explanation strategies affect user trust, and further explored user preferences regarding different XAI methods. In conclusion, this thesis advances the state of the art in both deep learning and XAI for assistive technologies aimed at elderly care. By addressing the dual challenges of performance and interpretability, our work contributes to the development of robust, transparent, and user-centered recognition systems. These systems not only have the potential to improve monitoring and intervention strategies—thereby enhancing the physical and emotional well-being of older adults—but also pave the way for future research in human-centric AI. Ultimately, the integration of effective FER and HAR with explainable AI represents a significant step toward creating safer, more autonomous living environments for the aging population, ensuring that technological innovations translate into tangible societal benefits.