To realize accurate environment perception, which is the technological key to enabling autonomous vehicles to interact with their external environments, it is primarily necessary to solve the issues of object detection and tracking in the vehicle-movement process. Multi-sensor fusion has become an essential process in efforts to overcome the shortcomings of individual sensor types and improve the efficiency and reliability of autonomous vehicles. This paper puts forward moving object detection and tracking methods based on LiDAR—camera fusion. Operating based on the calibration of the camera and LiDAR technology, this paper uses YOLO and PointPillars network models to perform object detection based on image and point cloud data. Then, a target box intersection-over-union (IoU) matching strategy, based on center-point distance probability and the improved Dempster–Shafer (D–S) theory, is used to perform class confidence fusion to obtain the final fusion detection result. In the process of moving object tracking, the DeepSORT algorithm is improved to address the issue of identity switching resulting from dynamic objects re-emerging after occlusion. An unscented Kalman filter is utilized to accurately predict the motion state of nonlinear objects, and object motion information is added to the IoU matching module to improve the matching accuracy in the data association process. Through self-collected data verification, the performances of fusion detection and tracking are judged to be significantly better than those of a single sensor. The evaluation indexes of the improved DeepSORT algorithm are 66% for MOTA and 79% for MOTP, which are, respectively, 10% and 5% higher than those of the original DeepSORT algorithm. The improved DeepSORT algorithm effectively solves the problem of tracking instability caused by the occlusion of moving objects.

This paper presents a VIDAR (a Vision-IMU based detection and ranging method)-based approach to road-surface pothole detection. Most potholes on the road surface are caused by the further erosion of cracks in the road surface, and tires, wheels and bearings of vehicles are damaged to some extent as they pass through the potholes. To ensure the safety and stability of vehicle driving, we propose a VIDAR-based pothole-detection method. The method combines vision with IMU to filter, mark and frame potholes on flat pavements using MSER to calculate the width, length and depth of potholes. By comparing it with the classical method and using the confusion matrix to judge the correctness, recall and accuracy of the method proposed in this paper, it is verified that the method proposed in this paper can improve the accuracy of monocular vision in detecting potholes in road surfaces.

Road obstacle detection is an important component of intelligent assisted driving technology. Existing obstacle detection methods ignore the important direction of generalized obstacle detection. This paper proposes an obstacle detection method based on the fusion of roadside units and vehicle mounted cameras and illustrates the feasibility of a combined monocular camera inertial measurement unit (IMU) and roadside unit (RSU) detection method. A generalized obstacle detection method based on vision IMU is combined with a roadside unit obstacle detection method based on a background difference method to achieve generalized obstacle classification while reducing the spatial complexity of the detection area. In the generalized obstacle recognition stage, a VIDAR (Vision-IMU based identification and ranging) -based generalized obstacle recognition method is proposed. The problem of the low accuracy of obstacle information acquisition in the driving environment where generalized obstacles exist is solved. For generalized obstacles that cannot be detected by the roadside unit, VIDAR obstacle detection is performed on the target generalized obstacles through the vehicle terminal camera, and the detection result information is transmitted to the roadside device terminal through the UDP (User Data Protocol) protocol to achieve obstacle recognition and pseudo-obstacle removal, thereby reducing the error recognition rate of generalized obstacles. In this paper, pseudo-obstacles, obstacles with a certain height less than the maximum passing height of the vehicle, and obstacles with a height greater than the maximum passing height of the vehicle are defined as generalized obstacles. Pseudo-obstacles refer to non-height objects that appear to be “patches” on the imaging interface obtained by visual sensors and obstacles with a height less than the maximum passing height of the vehicle. VIDAR is a vision-IMU-based detection and ranging method. IMU is used to obtain the distance and pose of the camera movement, and through the inverse perspective transformation, it can calculate the height of the object in the image. The VIDAR-based obstacle detection method, the roadside unit-based obstacle detection method, YOLOv5 (You Only Look Once version 5), and the method proposed in this paper were applied to outdoor comparison experiments. The results show that the accuracy of the method is improved by 2.3%, 17.4%, and 1.8%, respectively, compared with the other four methods. Compared with the roadside unit obstacle detection method, the speed of obstacle detection is improved by 1.1%. The experimental results show that the method can expand the detection range of road vehicles based on the vehicle obstacle detection method and can quickly and effectively eliminate false obstacle information on the road.

With the rapid progress of intelligent vehicle technology, the accurate recognition of road surface types and conditions has emerged as a crucial technology for improving the safety and comfort levels in autonomous driving. This paper puts forward a multi-feature fusion approach for road surface identification. Relying on a 24 GHz millimeter-wave radar, statistical features are combined with wavelet transform techniques. This combination enables the efficient classification of diverse road surface types and conditions. Firstly, the discriminability of radar echo signals corresponding to different road surface types is verified via statistical analysis. During this process, six-dimensional statistical features that display remarkable differences are extracted. Subsequently, a novel radar data reconstruction approach is presented. This method involves fitting discrete echo signals into coordinate curves. Then, discrete wavelet transform is utilized to extract both low-frequency and high-frequency features, thereby strengthening the spatio-temporal correlation of the signals. The low-frequency information serves to capture general characteristics, whereas the high-frequency information reflects detailed features. The statistical features and wavelet transform features are fused at the feature level, culminating in the formation of a 56-dimensional feature vector. Four machine learning models, namely the Wide Neural Network (WNN), K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Kernel methods, are employed as classifiers for both training and testing purposes. Experiments were executed with 8865 samples obtained from a real-vehicle platform. These samples comprehensively represented 12 typical road surface types and conditions. The experimental outcomes clearly indicate that the proposed method is capable of attaining a road surface type identification accuracy as high as 94.2%. As a result, it furnishes an efficient and cost-efficient road perception solution for intelligent driving systems. This research validates the potential application of millimeter-wave radar in intricate road environments and offers both theoretical underpinning and practical support for the advancement of autonomous driving technology.

Traffic-sign detection and recognition (TSDR) is crucial to avoiding harm to pedestrians, especially children, from intelligent connected vehicles and has become a research hotspot. However, due to motion blurring, partial occlusion, and smaller sign sizes, pedestrian TSDR faces increasingly significant challenges. To overcome these difficulties, a CTM-YOLOv8n model is proposed based on the YOLOv8n model. With the aim of extracting spatial features more efficiently and making the network faster, the C2f Faster module is constructed to replace the C2f module in the head, which applies filters to only a few input channels while leaving the remaining ones untouched. To enhance small-sign detection, a tiny-object-detection (TOD) layer is designed and added to the first C2f layer in the backbone. Meanwhile, the seventh Conv layer, eighth C2f layer, and connected detection head are deleted to reduce the quantity of model parameters. Eventually, the original CIoU is replaced by the MPDIoU, which is better for training deep models. During experiments, the dataset is augmented, which contains the choice of categories ‘w55’ and ‘w57’ in the TT100K dataset and a collection of two types of traffic signs around the schools in Tianjin. Empirical results demonstrate the efficacy of our model, showing enhancements of 5.2% in precision, 10.8% in recall, 7.0% in F1 score, and 4.8% in mAP@0.50. However, the number of parameters is reduced to 0.89M, which is only 30% of the YOLOv8n model. Furthermore, the proposed CTM-YOLOv8n model shows superior performance when tested against other advanced TSDR models.