PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
June 1, 2026IEEE Transactions on Image Processing8 citations

Decouple-Then-Synergize: A Self-Paced Collaborative Learning Network for RGB-T Snowy Urban Scene Parsing

View Full Paper
WZWujie ZhouYLYiben LiQJQiuping Jiang

Key Points

  • The aim is to enhance urban scene analysis by improving the integration of RGB and thermal infrared images under snowy conditions.
  • Proposed a 'decouple-then-synergize' framework using FRENet and SIFNet.
  • FRENet employs an asymmetric enhancement strategy for RGB gradients and thermal targets.
  • Implemented a self-paced curriculum for bidirectional knowledge exchange between FRENet and SIFNet.
  • The framework outperformed existing state-of-the-art scene parsing methods on SUS and PST900 datasets.

Abstract

Fusing RGB and thermal infrared images is essential for advancing urban scene analysis. However, both modalities exhibit severe performance degradation under snowy conditions. Although independent enhancement modules can partially mitigate this issue, stacking multiple modules with different functions increases model complexity and may cause intermodular interference. To address these limitations, we propose a "decouple-then-synergize" framework that decouples the task into frequency-oriented enhancement and spatial semantic fusion, implemented by FRENet (frequency restoration enhancement network) and SIFNet (spatial interactive fusion network), respectively. FRENet uses an asymmetric enhancement strategy that selectively sharpens RGB color gradients while amplifying faint thermal targets. It incorporates a precise spectral refinement module to restore high-frequency details. SIFNet introduces a Mamba zipper fusion module to achieve robust interaction of high-level semantics and performs a reconstruction task to implicitly integrate thermal features into the RGB stream. To ensure effective collaboration between the two networks, we design a self-paced curriculum that manages bidirectional knowledge exchange at both the sample and pixel levels. This approach enables the networks to evolve into their enhanced versions, namely FRENet-collaborative learning (CL) and SIFNet-CL. Extensive experiments on the SUS and PST900 datasets demonstrate that our framework outperforms state-of-the-art scene parsing methods. The code and associated results are available at https://github.com/Lyb-2001/SPCL.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Zhou et al. (2026) studied this question.

synapsesocial.com/papers/6a1d218f02fbce9130637906https://doi.org/10.1109/tip.2026.3696109
Ask AI
Helpful
Bookmark
Share
View Full Paper