面部动作单元检测的时空融合网络

Key Points

• The main aim is to develop a unified framework for detecting facial action units (AUs) by integrating both spatial and temporal information.
• Utilized ResNet-18 for feature extraction.
• Designed Spatial Relationship Modeling (SRM) and Temporal Relationship Modeling (TRM) modules.
• SRM models intra-frame AU interactions using graph neural networks.
• TRM captures dynamic changes through inter-frame differences and graph modeling.
• Implemented a temporal-spatial feature fusion strategy to balance spatial and temporal importance.
• Achieved an average F1-score of 66.00% on the BP4D dataset, outperforming recent methods.
• On the DISFA dataset, attained an average F1-score of 65.34%, nearing the optimal result.
• Ablation studies highlighted the significance of SRM, TRM, and SFF collaborations for effective AU detection.

Abstract