10 November 2020 in IEEE Transactions on Intelligent Transportation Systems
This article presents a semi-automated method to extract the lane features along the curved roads from mobile laser scanning (MLS) point clouds. The proposed method consists of four steps. After data pre-processing, a road edge detection algorithm is performed to distinguish road curbs and extract road surfaces. Then, textual and directional road markings such as arrows, symbols, and words, to inform drivers in necessary cases, are detected by intensity thresholding and conditional Euclidean clustering algorithms. Furthermore, lane markings are extracted by local intensity analysis and distance thresholding methods according to road design standards, because they are more regular along the road. Finally, centerline points on lanes are estimated based on the coordinates of extracted lane markings. Our method shows strong feasibility and robustness when creating high-definition (HD) maps from MLS data, by increasing the number of blocks in the curve and the distance threshold control in curved lane centerline extraction. Quantitative evaluations show that the average recall, precision, and F1-score obtained from four datasets for road marking extraction are 93.87%, 93.76%, and 93.73%, respectively. The generated lane centerlines are evaluated by overlaying them on manually labeled reference buffers from 4 cm resolution orthoimagery. The comparative study indicates that the proposed methods can achieve higher accuracy and robustness than most state-of-the-art methods.
30 November 2021 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Point cloud filtering is a preliminary and essential step in various applications of airborne LiDAR (light detection and ranging) data, with progressive triangulated irregular network (TIN) densification (PTD) being one of the classic methods for filtering LiDAR point clouds. The PTD algorithm densifies ground points through iteration operation based on initial ground seed points. However, the poor performance in steeply sloped areas and time-consuming processing are serious drawbacks for PTD algorithms. In this paper, we propose a fast progressive TIN densification (FPTD) filtering algorithm for airborne LiDAR data using adjacent surface information. After carefully establishing parameters and removing outliers, our improved FPTD uses a sliding window to obtain significantly more initial ground seed points. And we modified some iterative determination criterion, including the definition of maximum relative elevation threshold and the introduction of signed computation, to eliminate avoidable non-ground points. Then adjacent surface information was utilized to iterate each point cloud block, which is the smallest unit that point cloud can be segmented. Additionally, the algorithm is easily run in a multi-threaded environment, further accelerating the filtering process to some extent. Experiments show that our proposed FPTD filtering algorithm is fast and robust. Compared to the PTD, the FPTD algorithm yields better error rates and kappa coefficients in 1/12 of the time required by the PTD.
17 April 2025 in IEEE Internet of Things Journal
Due to the increase of signal bandwidth, automotive radar can receive multiple scatting centers from the target of interest. This is typically described by an extended target model. However, existing methods for extended target detection and tracking often exhibit subpar performance because they struggle to adapt to the hybrid-scale and time-varying texture feature exhibited by moving traffic participants. Therefore, this work presents a unified framework hybrid-scale spatio-temporal transformer for joint detection and tracking of hybrid-scale extended targets. It consists of three progressive stages: 1) hybrid-scale embedding network (HEN); 2) spatio-temporal transformer; and 3) Wasserstein loss. The HEN extracts the features of hybrid-scale targets by parallel multiple convolution kernels of different scales. Then, the spatio-temporal transformer locates targets by integrating these spatial-temporal features across the time. It can allow the target’s hybrid-scale and time-varying texture feature changes incurred through the movement, while eliminating the need for cross frame target association, thus achieving robust detection and tracking. The Wasserstein loss metric is designed to assist the model in optimizing hybrid-scale target location proficiency, especially for small targets. With intensive experiments on simulated datasets and public real datasets, such as RAMP-convolutional neural network and RADDet, the proposed model achieves better detection and tracking performance than existing methods with different noise added, and shows strong robustness in various traffic scenarios.
15 October 2025 in IEEE Transactions on Intelligent Transportation Systems
Radar and vision fusion in autonomous driving has gained increasing attention, however, most existing feature-level fusion methods (named as feature map-based fusion) typically map sparse point clouds into dense feature maps, fuse them with visual features, and then apply the combined features to detect sparse objects. These methods face challenges due to the structural disparity between sparse point clouds and dense feature maps, and memory-redundant mapping of point cloud processing. This paper proposes a proposal-guided radar and vision fusion framework for 3D object detection in autonomous driving, where proposals are referred to as potential objects with abundant high dimensional features. Specifically, two frame radar point clouds are first accumulated by deep learning-based motion compensation with supervision. Then, proposals are generated from radar point clouds and vision images. Subsequently, radar and vision fusion is performed on the proposal level using attention mechanism. Finally, objects are detected based on the fused proposal feature. The proposed framework introduces an object-centric feature fusion methodology that skips the common process of point-to-feature map projection, thereby circumventing the inherent structural disparity between sparse point clouds and dense feature maps. Experiments on two different autonomous driving datasets (nuScenes and VoD) demonstrate that our fusion method can outperform most published radar and vision fusion networks. The source code will be released at https://github.com/leehungu/prv
25 December 2024 in IEEE Sensors Journal
Association is crucial but easy to be neglected in radar and vision object-level fusion. Measurement uncertainty, consisting of complementary random errors of heterogeneous sensors and unpredictable deviation error due to monocular vision ranging, impedes a powerful radar and vision association. This paper proposes a robust radar and vision association method for measurement uncertainty and loss. First, radar and vision measurements are obtained by data preprocessing. Then, the cost matrix is initialized using the weighted Euclidean distance based on measurement random errors and road lane constraints. Next, the initial cost matrix is adaptively modified based on vision measurement calibration model to respond to vision deviation error. Finally, Hungarian algorithm is enhanced to tackled the unbalanced assignment problem caused by measurement loss. Experiments under different environmental conditions (clear weather, rain, and nighttime) show the average total correct association rate of the proposed method is at least 5.24% higher than comparative methods, indicating a more reliable association performance, especially in the distant region.