AI-Boosted Affective Real-Time Educational Software Adaptation

Nowadays, educational software across all learning levels is increasingly enhanced with Artificial Intelligence (AI), primarily through content generation or post-session learning analytics. However, most existing systems remain weakly connected to learners’ real-time affective states and rarely exploit emotional information as a direct control signal for instructional adaptation. In this work, we propose a proof-of-concept closed-loop affect-aware educational adaptation framework that integrates real-time facial emotion recognition into a dynamic learning control system. The proposed approach is built upon a dual-model ensemble architecture, combining a transformer-based model (CAGE) and a CNN-based model (DDAMFN++) trained on large-scale in-the-wild datasets. To bridge heterogeneous emotion representations, we introduce a probabilistic fusion strategy that aligns continuous valence–arousal predictions with discrete emotion classification via a Gaussian Mixture Model (GMM), enabling unified emotion inference in real time. Based on the fused emotional state, a temporal aggregation mechanism is applied to capture sustained affective trends rather than transient expressions. These aggregated signals are then mapped to instructional decisions through an emotion-driven adaptive control policy, which adjusts activity difficulty using an Average Emotion Score (AES). This establishes a fully automated closed-loop adaptation cycle, where detected learner affect directly influences the learning environment without requiring explicit user input or post-session questionnaires. The framework is integrated into an open-source educational platform (eduActiv8) to demonstrate feasibility and system-level behavior. Results from alpha-level validation show that the system can continuously monitor learner affect, generate interpretable emotional analytics, and dynamically adjust task difficulty in real time, while reducing user interaction overhead. This study contributes a modular architecture for affect-aware educational systems by combining real-time ensemble emotion recognition, probabilistic fusion of heterogeneous outputs, and closed-loop instructional adaptation. The proposed framework provides a foundation for future research in scalable, emotion-driven intelligent tutoring and adaptive learning environments.