Autonomous driving (AD) endows vehicles with the capability to drive partly or entirely without human intervention. AD agents generate driving policies based on online perception results, which are crucial to the realization of safe, efficient, and comfortable driving behaviors, particularly in high-dimensional and stochastic traffic scenarios. Currently, deep reinforcement learning (DRL) techniques to derive and validate AD policies have witnessed vast research efforts and have shown rapid development in recent years. However, a comprehensive interpretation and evaluation of their strengths and limitations concerning the full-stack AD tasks remain uncharted. This paper presents a survey of this body of work, which is conducted at three levels. First, it analyzes the multi-level AD task characteristics and delves deeply into the current DRL methodologies primarily employed in AD. Second, a taxonomy of the literature studies is constructed from the system perspective, identifying six modes of DRL model integration into an AD architecture that span the entire spectrum of AD policy processes, from perception understanding and decision-making to motion control, as well as verification and validation. Each literature review comprehensively encompasses the main elements of designing such a system, including modeling partially observable environments, state and action spaces, reward structuring, and the design and training methodologies of neural network models. Finally, an in-depth foresight is conducted on how the eight critical issues of AD application development are addressed by the DRL models tailored for real-world AD challenges.

Highly integrated information sharing among people, vehicles, roads, and cloud systems, along with the rapid development of autonomous driving technologies, has spurred the evolution of automobiles from simple “transportation tools” to interconnected “intelligent systems”. The intelligent cockpit is a comprehensive application space for various new technologies in intelligent vehicles, encompassing the domains of driving control, riding comfort, and infotainment. It provides drivers and passengers with safety, comfort, and pleasant driving experiences, serving as the gateway for traditional automobile manufacturing to upgrade towards an intelligent automotive industry ecosystem. This is the optimal convergence point for the intelligence, connectivity, electrification, and sharing of automobiles. Currently, the form, functions, and interaction methods of the intelligent cockpit are gradually changing, transitioning from the traditional “human adapts to the vehicle” viewpoint to the “vehicle adapts to human”, and evolving towards a future of natural interactive services where “humans and vehicles mutually adapt”. This article reviews the definitions, intelligence levels, functional domains, and technical frameworks of intelligent automotive cockpits. Additionally, combining the core mechanisms of human–machine interactions in intelligent cockpits, this article proposes an intelligent-cockpit human–machine interaction process and summarizes the current state of key technologies in intelligent-cockpit human–machine interactions. Lastly, this article analyzes the current challenges faced in the field of intelligent cockpits and forecasts future trends in intelligent cockpit technologies.

Autonomous Intersection Management (AIM) systems present a new paradigm for conflict-free cooperation of connected autonomous vehicles (CAVs) at road intersections, the aim of which is to eliminate collisions and improve the traffic efficiency and ride comfort. Given the challenges of current centralized coordination methods in balancing high computational efficiency and robust safety assurance, this paper proposes an innovative conflict-free management scheme for CAVs at unsignalized intersections, leveraging safe multi-agent deep reinforcement learning (MADRL). Firstly, we formulate the safe MADRL problem as a constrained Markov game (CMG) and then transform the AIM problem into a CMG by carefully designing state, action, reward, and cost functions. Subsequently, we propose the Multi-Agent Constrained Policy Optimization (MACPO), specifically tailored to solve the CMG problem. MACPO incorporates safety constraints that further restrict the trust region formed by the Kullback-Leibler (KL) divergence, facilitating reinforcement learning policy updates that maximize performance while keeping constraint costs within their limit bounds. This leads us to introduce the MACPO-based AIM Algorithm. Finally, we train an AIM policy and compare its computation time, ride comfort, traffic efficiency, and safety with management schemes based on Model Predictive Control (MPC), Mixed Integer Programming (MIP), and non-safety-aware reinforcement learning. According to the results, compared with the MPC and MIP methods, our method has increased computational efficiency by 65.22 times and 731.52 times respectively, and has improved traffic efficiency by 2.41 times and 1.80 times respectively. In contrast to the non-safety awareness RL methods, our method achieves a zero collision rate for the first time, while also enhancing ride comfort, highlighting the advantages of using MACPO.

With the recent advancements in machine learning technology, the accuracy of autonomous driving object detection models has significantly improved. However, due to the complexity and variability of real-world traffic scenarios, such as extreme weather conditions, unconventional lighting, and unknown traffic participants, there is inherent uncertainty in autonomous driving object detection models, which may affect the planning and control in autonomous driving. Thus, the rapid and accurate quantification of this uncertainty is crucial. It contributes to a better understanding of the intentions of autonomous vehicles and strengthens trust in autonomous driving technology. This research pioneers in quantifying uncertainty in the YOLOv5 object detection model, thereby improving the accuracy and speed of probabilistic object detection, and addressing the real-time operational constraints of current models in autonomous driving contexts. Specifically, a novel probabilistic object detection model named M-YOLOv5 is proposed, which employs the MC-drop method to capture discrepancies between detection results and the real world. These discrepancies are then converted into Gaussian parameters for class scores and predicted bounding box coordinates to quantify uncertainty. Moreover, due to the limitations of the Mean Average Precision (MAP) evaluation metric, we introduce a new measure, Probability-based Detection Quality (PDQ), which is incorporated as a component of the loss function. This metric simultaneously assesses the quality of label uncertainty and positional uncertainty. Experiments demonstrate that compared to the original YOLOv5 algorithm, the M-YOLOv5 algorithm shows a 74.7% improvement in PDQ. When compared with the most advanced probabilistic object detection models targeting the MS COCO dataset, M-YOLOv5 achieves a 14% increase in MAP, a 17% increase in PDQ, and a 65% improvement in FPS. Furthermore, against the state-of-the-art probabilistic object detection models for the BDD100K dataset, M-YOLOv5 exhibits a 31.67% enhancement in MAP and a 125.6% increase in FPS.

Accurate trajectory prediction for multiple vehicles in complex social interaction environments is essential for ensuring the safety of autonomous vehicles and improving the quality of their planning and control. The social interactions between vehicles significantly influence their future trajectories. However, traditional trajectory prediction models based on Recurrent Neural Networks (RNN) or Convolutional Neural Networks (CNN) often overlook or simplify these interactions. Although these models may exhibit high performance in short-term predictions, they fail to achieve high prediction accuracy in scenarios with long-term dynamic interactions. To address this limitation, we propose a Social-Attention Long Short-Term Memory (LSTM) model which predicts the future trajectories of neighboring vehicles and achieves increased accuracy. Our proposed model employs a Social-Pooling layer to effectively capture cooperative behaviors and mutual influences between vehicles. Additionally, we incorporate a self-attention mechanism to weight the inputs and outputs of the Social-Pooling layer, which is significant for assessing the influence between vehicles in different positions. This combination allows our model to take into consideration both the dependencies within the sequence and the social relationships between vehicles, providing a more comprehensive scene understanding. The efficacy of our model is tested on two real-world freeway trajectory datasets, namely NGSIM and HighD. Our model surpasses various baseline methods, exhibiting exceptional accuracy in both prediction and tracking.

Although reinforcement learning (RL) methodologies exhibit potential in addressing decision-making and planning problems in autonomous driving, ensuring the safety of the vehicle under all circumstances remains a formidable challenge in practical applications. Current RL methods are predominantly driven by singular reward mechanisms, frequently encountering difficulties in balancing multiple sub-rewards such as safety, comfort, and efficiency. To address these limitations, this paper introduces a constraint-driven safety RL method, applied to decision-making and planning policy in highway scenarios. This method ensures decisions maximize performance rewards within the bounds of safety constraints, exhibiting exceptional robustness. Initially, the framework reformulates the autonomous driving decision-making problem as a Constrained Markov Decision Process (CMDP) within the safety RL framework. It then introduces a Multi-Level Safety-Constrained Policy Optimization (MLSCPO) method, incorporating a cost function to address safety constraints. Ultimately, simulated tests conducted within the CARLA environment demonstrate that the proposed method MLSCPO outperforms the current advanced safe reinforcement learning policy, Proximal Policy Optimization with Lagrangian (PPO-Lag) and the traditional stable longitudinal and lateral autonomous driving model, Intelligent Driver Model with Minimization of Overall Braking Induced by Lane Changes (IDM+MOBIL). Compared to the classic IDM+MOBIL method, the proposed approach not only achieves efficient driving but also offers a better driving experience. In comparison with the reinforcement learning method PPO-Lag, it significantly enhances safety while ensuring driving efficiency, achieving a zero-collision rate. In the future, we will integrate the aforementioned potential expansion plans to enhance the usability and generalization capabilities of the method in real-world applications.