Multimodal fusion-based object detection is the foundational sensing task in scene understanding. It capitalizes on LiDAR and camera data to boost the robust results. However, there are still great challenges in establishing an effective fusion mechanism and performing accurate and diverse feature interaction fusion. In particular, the relationship construction between the two modalities has not been comprehensively exploited, leading to sensor data utilization deficiencies and redundancies. In this paper, a novel 3D object-detection framework, namely a symmetry-aware sparse sensor fusion detection network (2SFNet), is proposed. This framework was designed to leverage point clouds and RGB images. The 2SFNet consists of three submodules, filtered colored point cloud generation, pseudo-image generation, and a dilated feature fusion network, to solve these problems. Firstly, filtered colored point cloud generation constructs non-ground colored point cloud (NCPC) data by employing an early fusion strategy and a ground-height-filtering module, selectively retaining only object-related information. Subsequently, 2D grid encoding is used on the reduced colored data. Finally, the processed colored data are fed into the improved PillarsNet architecture, which now has expanded receptive fields to enhance the fusion effect. This design optimizes the fusion process by ensuring a more balanced and effective data representation, aligning with the symmetry concept that underlies the model’s functionality. Experiments and evaluations were conducted on the KITTI dataset to present the effectuality, particularly for categories characterized by sparse point clouds. The results indicate that the symmetry-aware design of the 2SFNet leads to an improved performance when compared to other multimodal fusion networks, and alleviates the phenomenon caused by highly obscured and crowded scenes.
Accurate road segmentation is a core perceptual technology for autonomous driving, but faces two challenges: (1) ambiguous road boundaries caused by insufficient modeling of contextual information relationships in CNN-based networks and (2) inadequate LiDAR-camera fusion due to modality gaps between heterogeneous sensors. To mitigate these limitations, this paper proposes a novel approach, named LiDAR-Camera High-Resolution Feature Fusion Network (LC-HR2FNet), a multi-cross-stage fusion model designed for road segmentation. Firstly, a new type of pseudo-LiDAR-Image representation is generated via an early-level fusion strategy and data complementation. Sparse point clouds are transformed into dense LiDAR-Image data and then concatenated with RGB channel maps to form complementary multi-modal data inputs. Subsequently, a modified HRNet backbone integrated with cross-stage feature fusion is constructed to strengthen information interaction across different branches and enhance the modeling of contextual relationships. Additionally, a dilated feature collection model is designed to collect multi-scale confidence scores for pixel-wise class determination. Experiments on the KITTI road benchmark demonstrate that the proposed method achieves a MaxF of 97.39% on UMM_ROAD and an average of 96.28% across all urban scenarios, demonstrating superior performance and robustness.