Accurate 3D object detection is vital in autonomous driving. Traditional LiDAR models struggle with sparse point clouds. We propose a novel approach integrating LiDAR and camera data to maximize sensor strengths while overcoming individual limitations for enhanced 3D object detection. Our research introduces the Channel-wise and Spatially-guided Multimodal feature fusion network (CSMNET) for 3D Object Detection. First, our method enhances LiDAR data by projecting it onto a 2D plane, enabling the extraction of class-specific features from a probability map. Second, we design class-based farthest point sampling (C-FPS), which boosts the selection of foreground points by utilizing point weights based on geometric or probability features while ensuring diversity among the selected points. Third, we developed a parallel attention-based multimodal fusion mechanism achieving higher resolution compared to raw LiDAR points. This fusion mechanism integrates two attention mechanisms: channel attention for LiDAR data and spatial attention for camera data. These mechanisms enhance the utilization of semantic features in a region of interest (ROI) to obtain more representative point features, leading to more effective fusion of information from both LiDAR and camera sources. Specifically, CSMNET achieves an Average Precision (AP) in Bird’s Eye View (BEV) detection of 90.16% (easy), 85.18% (moderate), and 80.51% (hard), with a mean AP (mAP) of 85.12%. In 3D detection, CSMNET attains 82.05% (easy), 72.64% (moderate), and 67.10% (hard) with a mAP of 73.75%. For 2D detection, the scores are 95.47% (easy), 93.25% (moderate), and 86.68% (hard), yielding a mAP of 91.72% for the KITTI dataset.

Autonomous vehicles (AVs) play a crucial role in enhancing urban mobility within the context of a smarter and more connected urban environment. Three-dimensional object detection in AVs is an essential task for comprehending the driving environment to contribute to their safe use in urban environments. Existing 3D LiDAR object detection systems lose many critical point features during the down-sampling process and neglect the crucial interactions between local features, providing insufficient semantic information and leading to subpar detection performance. We propose a dynamic feature abstraction with self-attention (DFA-SAT), which utilizes self-attention to learn semantic features with contextual information by incorporating neighboring data and focusing on vital geometric details. DFA-SAT comprises four modules: object-based down-sampling (OBDS), semantic and contextual feature extraction (SCFE), multi-level feature re-weighting (MLFR), and local and global features aggregation (LGFA). The OBDS module preserves the maximum number of semantic foreground points along with their spatial information. SCFE learns rich semantic and contextual information with respect to spatial dependencies, refining the point features. MLFR decodes all the point features using a channel-wise multi-layered transformer approach. LGFA combines local features with decoding weights for global features using matrix product keys and query embeddings to learn spatial information across each channel. Extensive experiments using the KITTI dataset demonstrate significant improvements over the mainstream methods SECOND and PointPillars, improving the mean average precision (AP) by 6.86% and 6.43%, respectively, on the KITTI test dataset. DFA-SAT yields better and more stable performance for medium and long distances with a limited impact on real-time performance and model parameters, ensuring a transformative shift akin to when automobiles replaced conventional transportation in cities.

3D object detection by autonomous vehicles is integral to intelligent transportation. Existing systems often compromise essential foreground point features and local spatial interactions through random down-sampling, focusing primarily on local feature extraction. However, this neglects interactions among distant yet significant points, limiting semantic information and detection performance due to inherent point cloud data sparsity. Addressing this, our proposed Geometric Feature Aggregation and Self-Attention in a Multi-Head Transformer (GFA-SMT) architecture leverages Graph Convolutional Networks and multi-channel transformers to enhance weak semantic information of distant sparse objects. GFA-SMT comprises three modules: Distance Suppression for Local Receptive Fields (DsLRF), Geometric Feature Aggregator with Multi-head Self Attention (GFaSA), and Predicted Key-point Weighting and Refinement (PKwR). DsLRF preserves foreground features, GFaSA encodes similar features and aggregates edge features, while PKwR focuses on key-points for enhancing geometric knowledge of distant and sparse objects. Extensive experiments on KITTI, DIARV2X-I and NuScenes datasets show significant enhancements in widely used techniques, resulting in notable increases in average precision (AP) for 3D object detection: 4.08%, 5.56%, and 4.62%, respectively, on the KITTI test dataset. GFA-SMT enhances point cloud detection accuracy, particularly at medium and long distances, with minimal impact on run-time performance and model parameters.