CA-GFNet: A Cross-Modal Adaptive Gated Fusion Network for Facial Emotion Recognition

Facial emotion recognition (FER) plays an important role in healthcare, human–computer interaction, and intelligent security systems. However, despite recent advances, many state-of-the-art FER methods depend on computationally intensive CNN or transformer backbones and large-scale annotated datasets while suffering noticeable performance degradation under cross-dataset evaluation because of domain shift. These limitations hinder practical usage in resource-constrained and real-world environments. To address this issue, we propose Cross-Adaptive Gated Fusion Network (CA-GFNet), a lightweight dual-stream FER framework that explicitly combines shallow structural features with deep semantic representations. The proposed architecture integrates domain-robust gradient-based descriptors with compact deep features extracted from a VGG-based backbone. After face detection and normalization, the structural stream captures fine-grained local appearance cues, whereas the semantic stream encodes high-level facial configurations. The two feature streams are projected into a shared latent space and adaptively fused using a gated fusion mechanism that learns sample-specific weights, allowing the model to prioritize the more reliable feature source under dataset shift. Extensive experiments on KDEF along with zero-shot cross-dataset evaluation on CK+ using a strict train-on-KDEF/test-on-CK+ protocol with subject-independent splits demonstrate the effectiveness of the proposed method. CA-GFNet achieves 99.30% accuracy on KDEF and 98.98% on CK+ while requiring significantly fewer parameters than conventional deep FER models. These results confirm that adaptive gated fusion of shallow and deep features can deliver both high recognition accuracy and strong cross-dataset robustness.