Semantic segmentation is a crucial technology for intelligent vehicles, enabling robust scene understanding in complex driving environments. However, existing methods often struggle with small, distant, and overlapping objects, posing challenges for safe autonomous operation. To address these limitations, we present FineSegNeRF, a model designed for fine semantic segmentation of such challenging scenarios. Our approach separates features along the depth dimension to perceive stereoscopic scene from spatial dimension, and then uses NeRF’s multi-view consistency to optimize the separated features for fine understanding. Meanwhile, a new “Semantic Uncertainty Neural Volume Render” method is proposed for constraining the consistency of volume density and semantic uncertainty estimation to further improve the semantic segmentation performance. Compared to current representative RGB-D and NeRF fusion semantic segmentation methods, our approach performs remarkable competitiveness in terms of fine semantic segmentation on VKITTI 2 and Replica datasets.
Liu et al. (Sun,) studied this question.