Cross-modal fusion driven 5D imaging system integrating spectral, polarization, and 3D modalities

Traditional multimodal imaging suffers from registration difficulties and weak information interaction due to the independent acquisition of separate modalities. Based on object-plane multiplexed encoding, this paper proposes a spectral–3D cooperative imaging method that integrates the complementary advantages of object-plane multiplexed encoding imaging (simultaneously acquiring spectral and coarse depth information) and polarization imaging (capturing approximate 3D shape and fine surface texture), thereby developing a new multi-dimensional imaging system capable of jointly acquiring spectral, 3D, and polarization information of the target. Hyperspectral information and coarse depth are obtained through an encoding–decoding mechanism, and a multi-scale gradient consistency sign correction (MS-GCSC) model is designed to use coarse depth as a before guide polarization-based 3D reconstruction. This approach effectively resolves azimuthal ambiguity and enables high-precision depth recovery. The system employs a single camera to synchronously capture polarization and encoded images, avoiding registration challenges and achieving simultaneous 3D-spectral-polarization measurement with an average spectral sampling interval of 3 nm and depth accuracy better than 0.1 mm. Experimental results verify that the proposed method achieves significant improvements in accuracy, stability, and adaptability.