3-D object detection is a vital part of outdoor scene perception. Learning the complete size and accurate positioning of objects from an incomplete point cloud spatial structure is essential to 3-D object detection. We propose a novel flexible 3-D heatmap auxiliary network (3-D HANet) for object detection. To obtain complete structure and location information from an incomplete point cloud structure, we propose a 3-D heatmap to reflect object information. Also, we design a plug-and-play auxiliary network based on 3-D heatmap, which improves the accuracy of the entire detection network without extra computation in the inference stage. We validate the 3-D HANet on the basis of three classic 3-D object detection networks: PointPillars, sparsely embedded convolutional detection (SECOND), and structure aware single-stage 3-D object detection from point cloud (SASSD). Experimental results show that our auxiliary network augments the feature extraction ability of the backbone network, which is manifested in that the predicted boxes and the ground-truth boxes are more suitable in size and more aligned in direction. Furthermore, we conducted verification experiments on the state-of-the-art (SOTA) detector, CasA, and made a further improvement on the official ranking of the KITTI dataset.
Hyperspectral imaging (HSI) contains abundant spectrums as well as spatial information, providing a great basis for classification in the field of remote sensing. In this paper, to make full use of HSI information, we combined spectral and spatial information into a two-dimension image in a particular order by extracting a data cube and unfolding it. Prior to the step of combining, principle component analysis (PCA) is utilized to decrease the dimensions of HSI so as to reduce computational cost. Moreover, the classification block used during the experiment is a convolutional neural network (CNN). Instead of using traditionally fixed-size kernels in CNN, we leverage a multi-scale kernel in the first convolutional layer so that it can scale to the receptive field. To attain higher classification accuracy with deeper layers, residual blocks are also applied to the network. Extensive experiments on the datasets from Pavia University and Salinas demonstrate that the proposed method significantly improves the accuracy in HSI classification.
15 December 2021 in IEEE Transactions on Intelligent Transportation Systems
One-stage 3D object detection using mobile light detection and ranging (LiDAR) has developed rapidly in recent years. Specifically, one-stage methods have attracted attention because of their high efficiency and light weight compared with two-stage methods. Inspired by this, we present the semantic segmentation assisted one-stage three-dimensional vehicle object detection (SSA3D), a network for the rapid detection of objects that keeps the advantages of the semantic segmentation module in the two-stage methods without increasing redundant computational load. First, we modified the sampling of the farthest point to improve the quality of the sampling points. This helps to reduce sampling outlier points and bad points that are difficult to perceive in the spatial structure information surrounding the point. Second, a neighbor attention group module is devoted to selectively add extra weight to neighbor points because of the different importance of neighbor points for the corresponding sampling point. Correctly increasing the weight is helpful to obtain richer spatial structure information. Finally, a delicate box generation module is included as a voted center point layer based on the generalized Hoff vote method and an anchor-free regression. We used the feature aggregation module as the backbone and the feature propagation module as the auxiliary network to achieve efficiency. At the same time, the auxiliary network retains the ability to extract point-wise features from the state-of-the-art semantic segmentation network. In experiments, we evaluated and tested the SSA3D on a common KITTI dataset and achieved improved performance in the class of car accuracy.
26 September 2024 in IEEE Transactions on Geoscience and Remote Sensing
LiDAR-based sparse 3D object detection plays a crucial role in autonomous driving applications due to its computational efficiency advantages. Existing methods either use the features of a single central voxel as an object proxy, or treat an aggregated cluster of foreground points as an object proxy. However, the former lacks the ability to aggregate contextual information, resulting in insufficient information expression in object proxies. The latter relies on multi-stage pipelines and auxiliary tasks, which reduce the inference speed. To maintain the efficiency of the sparse framework while fully aggregating contextual information, in this work, we propose SparseDet which designs sparse queries as object proxies. It introduces two key modules, the Local Multi-scale Feature Aggregation (LMFA) module and the Global Feature Aggregation (GFA) module, aiming to fully capture the contextual information, thereby enhancing the ability of the proxies to represent objects. Where LMFA module achieves feature fusion across different scales for sparse key voxels via coordinate transformations and using nearest neighbor relationships to capture object-level details and local contextual information, GFA module uses self-attention mechanisms to selectively aggregate the features of the key voxels across the entire scene for capturing scene-level contextual information. Experiments on nuScenes and KITTI demonstrate the effectiveness of our method. Specifically, SparseDet surpasses the previous best sparse detector VoxelNeXt (a tipical method using voxels as object proxies) by 2.2% mAP with 13.5 FPS on nuScenes, and outperforms VoxelNeXt by 1.12% AP 3D on hard level tasks with 17.9 FPS on KITTI. What’s more, not only the mAP of SparseDet exceeds that of FSDV2 (a classical method using clusters of foreground points as object proxies), but also its inference speed is 1.3 times faster than FSDV2 on nuScenes test set. The code has been released in https://github.com/liulin813/SparseDet.git.
27 May 2025 in IEEE Transactions on Intelligent Transportation Systems
3D object detection is an essential scene perception capability for autonomous vehicles. In intelligent transportation systems, autonomous vehicles require minimal inference latency to sense their surroundings in real-time. However, advanced 3D detection methods often suffer from high inference latency. This limits the real-time deployment of 3D detection models in the real world. To address this problem, this paper proposes a fast and deployable 3D object detection method from the LiDAR point cloud for autonomous driving, named Fade3D. Firstly, we propose a Lightweight Input Encoder (LIE) to extract the most critical features from point clouds. Then, we develop a Spatial Feature Enhancement BEV backbone (SFENet) that efficiently encodes geometry features into compact representations. Additionally, we design an IoU-aware Loss Re-weighting (ILR) that enhances performance by shifting more attention to hard samples. Leveraging LIE and SFENet, our approach is independent of point cloud density and number, achieving significant speed advantages in processing large-scale point clouds and being deployment-friendly. Extensive experiments on KITTI and Waymo Open Dataset (WOD) datasets comparing various baseline detectors demonstrate its universality and superiority. Specifically, our method demonstrates impressive real-time inference capabilities, achieving 51.5 Hz on an RTX3090 GPU and 12.4 Hz on a Jetson Orin embedded development board. Code will be available at https://github.com/wayyeah/Fade3D
3 November 2025 in IEEE Transactions on Circuits and Systems for Video Technology
As a critical task in autonomous driving perception systems, 3D object detection is used to identify and track key objects, such as vehicles and pedestrians. However, detecting distant, small, or occluded objects (hard instances) remains a challenge, which directly compromises the safety of autonomous driving systems. We observe that existing multi-modal 3D object detection methods often follow a single-guided paradigm, failing to account for the differences in information density of hard instances between modalities. In this work, we propose DGFusion, based on the Dual-guided paradigm, which fully inherits the advantages of the Point-guide-Image paradigm and integrates the Image-guide-Point paradigm to address the limitations of the single paradigms. The core of DGFusion, the Difficulty-aware Instance Pair Matcher (DIPM), performs instance-level feature matching based on difficulty to generate easy and hard instance pairs, while the Dual-guided Modules exploit the advantages of both pair types to enable effective multi-modal feature fusion. Experimental results demonstrate that our DGFusion outperforms the baseline methods, with respective improvements of +1.0% mAP, +0.8% NDS, and +1.3% average recall on nuScenes. Extensive experiments demonstrate consistent robustness gains for hard instance detection across ego-distance, size, visibility, and small-scale training scenarios.