Camera and millimeter-wave (MMW) radar fusion is essential for accurate and robust autonomous driving systems. With the advancement of radar technology, next-generation high-resolution automotive radar, i.e., 4D radar, has emerged. In addition to the target range, azimuth, and Doppler velocity measurements of traditional radar, 4D radar provides elevation measurement to create a denser “point cloud.” In this study, we propose a camera and 4D radar fusion network called RCFusion, which achieves multimodal feature fusion under a unified bird’s-eye view (BEV) space to accomplish 3D object detection tasks. In the camera stream, multi-scale feature maps are obtained by the image backbone and feature pyramid network; they are then converted into orthographic feature maps by an orthographic feature transform. Next, enhanced and fine-grained image BEV features are obtained via a designed shared attention encoder. Meanwhile, in the 4D radar stream, a newly designed component named Radar PillarNet efficiently encodes the radar features to generate radar pseudo-images, which are fed into the point cloud backbone to create radar BEV features. An interactive attention module is proposed for the fusion stage, which outputs a valid fusion of the two-modal BEV features. Finally, a generic detection head predicts the object classes and locations. The proposed RCFusion is validated on the TJ4DRadSet and View-of-Delft datasets. The experimental results and analysis show that the proposed method can effectively fuse camera and 4D radar features to achieve robust detection performance.

Automotive millimeter-wave (MMW) radar is essential in autonomous vehicles due to its robustness in all weather conditions. Traditional commercial automotive radars are limited by their resolution, which makes the object classification task difficult. Thus, the concept of a new generation of four-dimensional (4D) imaging radar was proposed. It has high azimuth and elevation resolution and contains Doppler information to produce a high-quality point cloud. In this paper, we propose an object classification network named Radar Transformer. The algorithm takes the attention mechanism as the core and adopts the combination of vector attention and scalar attention to make full use of the spatial information, Doppler information, and reflection intensity information of the radar point cloud to realize the deep fusion of local attention features and global attention features. We generated an imaging radar classification dataset and completed manual annotation. The experimental results show that our proposed method achieved an overall classification accuracy of 94.9%, which is more suitable for processing radar point clouds than the popular deep learning frameworks and shows promising performance.

Intelligent transportation systems (ITSs) play an increasingly important role in traffic management and traffic safety. Smart cameras are the most widely used sensors in ITSs. However, cameras suffer from a reduction in detection and positioning accuracy due to target occlusion and external environmental interference, which has become a bottleneck restricting ITS development. This work designs a stable perception system based on a millimeter-wave radar and camera to address these problems. Radar has better ranging accuracy and weather robustness, which is a better complement to camera perception. Based on an improved Gaussian mixture probability hypothesis density (GM-PHD) filter, we also propose an optimal attribute fusion algorithm for target detection and tracking. The algorithm selects the sensors’ optimal measurement attributes to improve the localization accuracy while introducing an adaptive attenuation function and loss tags to ensure the continuity of the target trajectory. The verification experiments of the algorithm and the perception system demonstrate that our scheme can steadily output the classification and high-precision localization information of the target. The proposed framework could guide the design of safer and more efficient ITSs with low costs.

This paper presents a target tracking algorithm based on 4D millimeter-wave radar point cloud information for autonomous driving applications, which addresses the limitations of traditional 2 + 1D radar systems by using higher resolution target point cloud information that enables more accurate motion state estimation and target contour information. The proposed algorithm includes several steps, starting with the estimation of the ego vehicle’s velocity information using the radial velocity information of the millimeter-wave radar point cloud. Different clustering suggestions are then obtained using a density-based clustering method, and correlation regions of the targets are obtained based on these clustering suggestions. The binary Bayesian filtering method is then used to determine whether the targets are dynamic or static targets based on their distribution characteristics. For dynamic targets, Kalman filtering is used to estimate and update the state of the target using trajectory and velocity information, while for static targets, the rolling ball method is used to estimate and update the shape contour boundary of the target. Unassociated measurements are estimated for the contour and initialized for the trajectory, and unassociated trajectory targets are selectively retained and deleted. The effectiveness of the proposed method is verified using real data. Overall, the proposed target tracking algorithm based on 4D millimeter-wave radar point cloud information has the potential to improve the accuracy and reliability of target tracking in autonomous driving applications, providing more comprehensive motion state and target contour information for better decision making.

Traffic participant classification is critical in autonomous driving perception. Millimetre wave radio detection and ranging (RADAR) is a cost‐effective and robust means of performing this task in adverse traffic scenarios such as inclement weather (e.g. fog, snow, and rain) and poor lighting conditions. Compared to commercial two‐dimensional RADAR, the new generation of three‐dimensional (3D) RADAR can obtain height information about targets as well as their dense point clouds, greatly improving target classification capabilities. This study proposes a multi‐objective classification method for traffic participants based on 3D RADAR point clouds. First, a 22‐dimensional feature vector of the 3D RADAR point cloud distribution was extracted to describe the shape, discrete, Doppler, and reflection intensity features of the targets. Then, dynamic and static datasets containing five classes of targets were produced, creating a 10k frame. Extensive experiments were conducted to build machine learning classifiers. The experimental results show that the trained classifiers can achieve over 92% classification accuracy when the targets are classified into five groups and over 95% classification accuracy when the targets are classified into four groups. The proposed method can guide the design of safer and more efficient intelligent driving systems.

In Advanced Driver Assistant System (ADAS), the automotive radar is used to detect targets or obstacles around the vehicle. The procedure of Constant False Alarm Rate (CFAR) plays an important role in adaptive targets detection in noise or clutter environment. But in practical applications, the noise or clutter power is absolutely unknown and varies over the change of range, time and angle. The well-known cell averaging (CA) CFAR detector has a good detection performance in homogeneous environment but suffers from masking effect in multi-target environment. The ordered statistic (OS) CFAR is more robust in multi-target environment but needs a high computation power. Therefore, in this paper, a new two-dimension CFAR procedure based on a combination of Generalized Order Statistic (GOS) and CA CFAR named GOS-CA CFAR is proposed. Besides, the Linear Frequency Modulation Continuous Wave (LFMCW) radar simulation system is built to produce a series of rapid chirp signals. Then the echo signals are converted into a two-dimensional Range-Doppler matrix (RDM), which contains information about the targets as well as background clutter and noise, through twice Fast Fourier Transform (FFT).The simulation experimental results show that compared to the two-dimensional OS-CA CFAR, the new 2-D GOS-CA CFAR can enhance the detection performance and robustness in the actual multi-target environment with lower computational complexity.