DAMCSeg: Dynamic Adaptive Multi-Modal Collaborative Semantic Segmentation

While current semantic segmentation models excel in controlled environments, they often struggle with key challenges such as dynamic multi-modal data, small target recognition, and computational efficiency for edge deployment. Motivated by these limitations, this study explores targeted solutions and presents DAMCSeg (Dynamic Adaptive Multi-modal Collaborative Semantic Segmentation), an innovative framework that introduces advancements across feature fusion, training paradigms, and model efficiency. The core contributions of DAM-CSeg include: 1) a Dual-Stage Attention Fusion (DSAF) module that dynamically adjusts multi-branch fusion weights based on scene complexity; 2) an end-to-end joint training framework for object detection and semantic segmentation designed to minimize inter-stage error propagation; and 3) a Lightweight Multi-Modal Fusion (LMMF) module that efficiently integrates multi-source data with low computational overhead. To rigorously evaluate the proposed method’s effectiveness against these specific challenges, extensive experiments are conducted on mainstream benchmark datasets. The results demonstrate that DAMCSeg achieves high accuracy and operational efficiency, effectively addressing critical issues in dynamic scene adaptation, complex target segmentation, and edge device deployment. This provides a practical and viable solution for semantic segmentation in demanding applications such as autonomous driving and medical image analysis.