The use of object detection algorithms has become extremely important in autonomous vehicles. Object detection at high accuracy and a fast inference speed is essential for safe autonomous driving. Therefore, the balance between the effectiveness and efficiency of the object detector must be considered. This article proposes a one-stage object detection framework for improving the detection accuracy while supporting a true real-time operation based on the YOLOv4. The backbone network in the proposed framework is the CSPDarknet53_dcn(P). The last output layer in the CSPDarknet53 is replaced with deformable convolution to improve the detection accuracy. In order to perform feature fusion, a new feature fusion module PAN++ is designed and five scales detection layers are used to improve the detection accuracy of small objects. In addition, this article proposes an optimized network pruning algorithm to solve the problem that the real-time performance of the algorithm cannot be satisfied due to the limited computing resources of the vehicle-mounted computing platform. The method of sparse scaling factor is used to improve the existing channel pruning algorithm. Compared to the YOLOv4, the YOLOV4-5D improves the mean average precision by 4.23% on the BDD data sets and 1.68% on the KITTI data sets. Finally, by pruning the model, the inference speed of YOLOV4-5D is increased 31.3% and the memory is only 98.1 MB when the detection accuracy is almost unchanged. Nevertheless, the proposed algorithm is capable of real-time detection at faster than 66 frames/s (fps) and shows higher accuracy than the previous approaches with a similar fps.

As self-driving vehicles become more prevalent, the speed and accuracy of detecting surrounding objects through onboard sensing technology have become increasingly important. The YOLOv8-QSD network is a novel anchor-free driving scene detection network that builds on YOLOv8 and ensures detection accuracy while maintaining efficiency. The network’s backbone employs structural reparameterization techniques to transform the Diverse Branch Block based model. To accurately detect small objects, it integrates features of different scales and implements a bifpn-based feature pyramid after the backbone. To address the challenge of long-range detection in driving scenarios, a query-based model with a new pipeline structure is introduced. The test results demonstrate that this algorithm outperforms YOLOv8 on the soda-a dataset in terms of both speed and accuracy. With an accuracy rate of 64.5% and reduced computational requirements of 7.1 GFLOPs, it satisfies the speed, precision, and cost-effectiveness requirements for commercial vehicles in high-speed road driving scenarios.

In vehicle trajectory prediction, the difficulty in modeling the interaction relationship between vehicles lies in constructing the interaction structure between the vehicles in the traffic scene. Majority of existing models only focus on the interaction between the historical trajectory of the vehicle and the surrounding vehicles in the spatial domain, and do not pay attention to the interaction between the vehicle and the non-Euclidean correlation structure (graph structure) that exists in the environment. In order to overcome the deficiencies in the existing models, this paper proposes the Environment-Attention Network model (EA-Net) to obtain the full interactive information between the vehicle and its driving environment. In the proposed model, a new type of parallel structure consisting of Graph Attention network (GAT) and Convolutional social pooling containing Squeeze-and-Extraction mechanism (SE-CS), is constructed as the environmental feature extraction module and embedded in LSTM encoder-decoder. This structure solves the limitation of the dimension and structure influence when constructing the interaction model between the vehicle and the surrounding environment, making the extracted feature information comprehensive and effective. The prediction accuracy of the model with the RMSE loss function is tested on two public datasets– NGSIM and highD, and compared with several state-of-the-art trajectory prediction algorithm models. The results show that the prediction accuracy of the proposed Environment-Attention Network in the two datasets is more than 20% higher than that of the single structure model, which indicates that the proposed model proposed has superior performance and better adaptability to different traffic environments compared with the existing models.

With the rise in popularity of autonomous driving, the speed and accuracy of surrounding objects' detection by in-vehicle sensing technology is becoming increasingly important for autonomous vehicles. Building on CenterNet, this paper proposes CenterNet-Auto, a new anchor-free detection network for driving scenes that can satisfy the detection speed requirements while ensuring detection accuracy. The network's backbone uses the RepVGG model transformed through structural re-parameterization technology. Features of different scales are fused, and feature pyramids and deformable convolution are added after the backbone to accurately detect objects of different sizes. To solve the occlusion problem in the driving scene, this paper proposes the Average Border Model, which supports locating the object using the boundary feature information. The test results demonstrate that the proposed algorithm outperforms CenterNet regarding speed and accuracy on the BDD dataset. The accuracy reaches 55.6%, and the speed reaches 30 FPS, meeting the speed and accuracy requirements in a driving scene.

Pedestrian motion trajectory prediction is an important task in intelligent driving, and it can provide a valuable reference for the subsequent path decision of intelligent driving. However, so far, there are only a few models in the field of specific pedestrian motion track prediction in intelligent driving from far shot first-person perspective video. To accomplish this task, we proposed a deep learning model for pedestrian motion trajectory prediction from far shot first-person perspective video with four key innovations: a) A macroscopic pedestrian trajectory prediction module is established under the close correlation between neighboring frames to estimate the pedestrian motion track on the whole; b) A relative motion transformation module of vehicle-mounted camera is designed to consider the effect of vehicle-mounted camera’s ego-motion on the pedestrian motion track; c) We set up a circular training module to maintain the number of parameters in our model to simplify and reduce the size of model; d) A new far shot first-person pedestrian motion dataset under intelligent driving is specifically established to train and test the proposed model. The above four modules are integrated into the proposed deep learning model, which achieves state-of-the-art results for predicting pedestrian motion trajectory from both far and close shot first-person perspective video.

With the rapid development of deep learning in recent years, the level of automatic driving perception has also increased substantially. However, automatic driving perception under adverse conditions, such as fog, remains a significant obstacle. The existing fog-oriented detection algorithms are unable to simultaneously address the detection accuracy and detection speed. Based on improved YOLOv5, this work provides a multiobject detection network for fog driving scenes. We construct a synthetic fog dataset by using the dataset of a virtual scene and the depth information of the image. Second, we present a detection network for driving in fog based on improved YOLOv5. The ResNeXt model, which has been modified by structural re-parameterization, serves as the model’s backbone. We build a new feature enhancement module (FEM) in response to the lack of features in fog scene images and use the attention mechanism to help the detection network pay more attention to the more useful features in the fog scenes. The test results show that the proposed fog multitarget detection network outperforms the original YOLOv5 in terms of detection accuracy and speed. The accuracy of the Real-world Task-driven Testing Set (RTTS) public dataset is 77.8%, and the detection speed is 31 frames/s, which is 14 frames faster as compared with the original YOLOv5.