Environmental perception with the multi-modal fusion is crucial in autonomous driving to increase accuracy, completeness, and robustness. This paper focuses on utilizing millimeter-wave (MMW) radar and camera sensor fusion for 3D object detection. A novel method that realizes the feature-level fusion under the bird's-eye view (BEV) for a better feature representation is proposed. Firstly, radar points are augmented with temporal accumulation and sent to a spatial-temporal encoder for radar feature extraction. Meanwhile, multi-scale image 2D features which adapt to various spatial scales are obtained by image backbone and neck model. Then, image features are transformed to BEV with the designed view transformer. In addition, this work fuses the multi-modal features with a two-stage fusion model called point-fusion and ROI-fusion, respectively. Finally, a detection head regresses objects category and 3D locations. Experimental results demonstrate that the proposed method realizes the state-of-the-art (SOTA) performance under the most crucial detection metrics–mean average precision (mAP) and nuScenes detection score (NDS) on the challenging nuScenes dataset.

With the rapid development of automated vehicles (AVs), more and more demands are proposed towards environmental perception. Among the commonly used sensors, MMW radar plays an important role due to its low cost, adaptability In different weather, and motion detection capability. Radar can provide different data types to satisfy requirements for various levels of autonomous driving. The objective of this study is to present an overview of the state-of-the-art radar-based technologies applied In AVs. Although several published research papers focus on MMW Radars for intelligent vehicles, no general survey on deep learning applied In radar data for autonomous vehicles exists. Therefore, we try to provide related survey In this paper. First, we introduce models and representations from millimeter-wave (MMW) radar data. Secondly, we present radar-based applications used on AVs. For low-level automated driving, radar data have been widely used In advanced driving-assistance systems (ADAS). For high-level automated driving, radar data is used In object detection, object tracking, motion prediction, and self-localization. Finally, we discuss the remaining challenges and future development direction of related studies.

The grid-centric perception is a crucial field for mobile robot perception and navigation. Nonetheless, the grid-centric perception is less prevalent than object-centric perception as autonomous vehicles need to accurately perceive highly dynamic, large-scale traffic scenarios, and the complexity and computational costs of grid-centric perception are high. In recent years, the rapid development of deep learning techniques and hardware provides fresh insights into the evolution of grid-centric perception. The fundamental difference between grid-centric and object-centric pipeline lies in that grid-centric perception follows a geometry-first paradigm which is more robust to the open-world driving scenarios with endless long-tailed semantically unknown obstacles. Recent research demonstrates the great advantages of grid-centric perception, such as comprehensive fine-grained environmental representation, greater robustness to occlusion and irregular-shaped objects, better ground estimation, and safer planning policies. There is also a growing trend that the capacity of occupancy networks is greatly expanded to 4-D scene perception and prediction, and the latest techniques are highly related to new research topics, such as 4-D occupancy forecasting, generative artificial intelligence (GenAI), and world models in the field of autonomous driving. Given the lack of current surveys for this rapidly expanding field, we present a hierarchically structured review of grid-centric perception for autonomous vehicles. We organize previous and current knowledge of occupancy grid techniques along the main vein from 2-D bird-eye view (BEV) grids to 3-D occupancy to 4-D occupancy forecasting. We additionally summarize label-efficient occupancy learning and the role of grid-centric perception in driving systems. Finally, we present a summary of the current research trend and provide future outlooks.

Occupancy prediction plays a crucial role in supporting autonomous driving planning and decision-making. Existing methods typically rely on modular stacking and fusion techniques of object detection, semantic segmentation, and depth estimation to achieve 3D occupancy. However, they fail to deeply explore the transformation relationships between 2D and 3D spaces and to efficiently fuse the different characteristics of multi-source sensors. We propose R $M^{2}$ Occ, the first 3D occupancy perception network that integrates multi-sensor fusion based on different sensor principles and achieves multi-task learning. To leverage the rich 2D semantic information captured by cameras and elevate it to the 3D domain, we begin by querying and populating predefined empty voxels with multi-view image features. Subsequently, we progressively fuse 3D LiDAR point clouds with these populated voxels through an unbalanced fusion strategy that effectively supplements missing information and suppresses noise. Leveraging IMU data and calibration parameters, we then re-project the enriched voxels back onto the 2D image plane according to camera coordinates, performing a secondary query using the semantic segmentation results to recover semantic details potentially lost due to radar fusion limitations and incomplete voxel querying. Finally, supported by a multi-task detection head, R $M^{2}$ Occ simultaneously accomplishes 3D object detection, semantic segmentation, Bird’s Eye View (BEV) detection, and full-scene grid occupancy prediction, enabling comprehensive multi-task output. Extensive experiments and ablation studies on the nuScenes dataset demonstrate that R $M^{2}$ Occ significantly outperforms existing state-of-the-art methods, establishing a new paradigm for accurate and efficient multi-sensor fusion and multi-task perception in autonomous driving scenarios.

In recent years, the field of autonomous vehicles has seen a significant increase in academic research, with high-definition (HD) maps emerging as a critical component of autonomous driving technology. These maps provide intricate details of road networks and serve as a fundamental input for critical tasks such as vehicle positioning, navigation and decision making. Given the widespread availability and affordability of camera sensors, they have become an indispensable aspect of autonomous vehicles, justifying their focus in this review. The aim of this article is to provide researchers with a comprehensive overview and summary of recent advances in HD mapping. The review begins with a concise summary of the key frameworks and background information relevant to the creation of HD maps using camera sensors. This is followed by a comprehensive review of the research methods used for map production, including both offline and online approaches. In particular, network-based mapping methods have emerged as a prominent area of research in the field of HD mapping. In response to this burgeoning trend, this article presents a comprehensive and meticulous overview of the diverse research efforts in this particular field. Finally, the article addresses pertinent issues and future challenges with the aim of guiding researchers in understanding the current trends and methodologies prevalent in the field.

Manhole covers, which are a key element of urban infrastructure management, have a direct impact on travel safety. At present, there is no automatic, safe, and efficient system specially used for the intelligent detection, identification, and assessment of manhole covers. In this work, we developed an automatic detection, identification, and assessment system for manhole covers. First, we developed a sequential exposure system via the addition of multiple cameras in a symmetrical arrangement to realize the joint acquisition of high-precision laser data and ultra-high-resolution ground images. Second, we proposed an improved histogram of an oriented gradient with symmetry features and a support vector machine method to detect manhole covers effectively and accurately, by using the intensity images and ground orthophotos that are derived from the laser points and images, respectively, and apply the graph segmentation and statistical analysis to achieve the detection, identification, and assessment of manhole covers. Qualitative and quantitative analyses are performed using large experimental datasets that were acquired with the modified manhole-cover detection system. The detected results yield an average accuracy of 96.18%, completeness of 94.27%, and F-measure value of 95.22% in manhole cover detection. Defective manhole-cover monitoring and manhole-cover ownership information are achieved from these detection results. The results not only provide strong support for road administration works, such as data acquisition, manhole cover inquiry and inspection, and statistical analysis of resources, but also demonstrate the feasibility and effectiveness of the proposed method, which reduces the risk involved in performing manual inspections, improves the manhole-cover detection accuracy, and serves as a powerful tool in intelligent road administration.