Artificial Intelligence (AI) shows promising applications for the perception and planning tasks in autonomous driving (AD) due to its superior performance compared to conventional methods. However, highly complex AI systems exacerbate the existing challenge of safety assurance of AD. One way to mitigate this challenge is to utilize explainable AI (XAI) techniques. To this end, we present the first comprehensive systematic literature review of explainable methods for safe and trustworthy AD. We begin by analyzing the requirements for AI in the context of AD, focusing on three key aspects: data, model, and agency. We find that XAI is fundamental to meeting these requirements. Based on this, we explain the sources of explanations in AI and describe a taxonomy of XAI. We then identify five key contributions of XAI for safe and trustworthy AI in AD, which are interpretable design, interpretable surrogate models, interpretable monitoring, auxiliary explanations, and interpretable validation. Finally, we propose a conceptual modular framework called SafeX to integrate the reviewed methods, enabling explanation delivery to users while simultaneously ensuring the safety of AI models.

Autonomous driving has attracted considerable attention from research and industry communities. Although prototypes of automated vehicles (AVs) are developed, remaining safety issues and functional insufficiencies hinder their market introduction. To obtain reasonably foreseeable scenarios and study human driving policies, many naturalistic driving datasets are proposed. However, no open-source dataset filled with congestion scenarios is publicly available. The paper presents the Aerial Dataset for China's Congested Highways & Expressways (AD4CHE). It contains 5.12 hours of aerial survey data from four different cities in China, with a total driving distance of 6540.7 km. Moreover, overlap and non-overlap cut-in scenarios are distinguished to better describe driver behavior in congestion scenarios. Both types of cut-in scenarios are extracted and parameterized. The Kernel Density Estimator (KDE) is utilized to generate parameter distributions for the scenario-based testing method. Furthermore, the driving behavior in overlap cut-in scenarios is intensively analyzed. The results reveal that the drivers have an evasive maneuver during overlap cut-in of challenging vehicles, and the preferred following distance varies with the relative longitudinal velocity. Both scenario parameterization and driving behavior analysis can contribute to developing and verifying Traffic Jam Pilot (TJP) systems deployed in Chinese traffic situations. The dataset is available at https://auto.dji.com/cn .

Driver models play a vital role in developing and verifying autonomous vehicles (AVs). Previously, they were mainly applied in traffic flow simulation to model driver behavior. With the development of AVs, driver models attract much attention again due to their potential contributions to AV safety assessment. The simulation-based testing method is an effective measure to accelerate AV testing due to its safe and efficient characteristics. Nonetheless, realistic driver models are prerequisites for valid simulation results. Additionally, an AV is assumed to be at least as safe as a careful and competent driver, which is modeled by driver models as well. Therefore, driver models are essential for AV safety assessment from the current perspective. However, no comparison or discussion of driver models is available regarding their utility to AVs in the last five years despite their necessities in the release of AVs. This motivates us to present a comprehensive survey of driver models in the paper and compare their applicability. Requirements for driver models as applied to AV safety assessment are discussed. A summary of driver models for simulation-based testing and AV benchmarks is provided. Evaluation metrics are defined to compare their strengths and weaknesses. Finally, potential gaps in existing driver models are identified, which provide direction for future work. This study gives related researchers especially regulators an overview and helps them to define appropriate driver models for AV safety assessment.

Safety validation is a challenge for releasing automated driving, even though substantial effort has been made in this field. This paper proposes an online safety validation method within the frame of virtual assessment of automation in field operation (VAAFO). The basic idea of VAAFO is that a virtual automated vehicle runs in the background, while the physical vehicle is driven in an automated or manual mode. The virtual automated vehicle receives input from real sensors, but has no access to the actuators. Thus, the risk-free nature of simulation-based testing and the validity of field operational testing are combined. The testing of automated vehicles is accelerated by applying the approach in customer vehicles. In this paper, we elaborate on and implement this approach. Essential parameters are studied and specified, while the necessary coordinate transformation is performed. Triggers are defined and concretized by a newly developed criticality index. Two derived simulated cases and a real-world case are utilized to illustrate the VAAFO approach. The study cases show that the safety of automated vehicles can be assessed online and critical scenarios can be discovered by the defined triggers under the framework of VAAFO. The results demonstrate that the proposed approach is able to safely and efficiently test automated vehicles online under real driving conditions and discover unknown unsafe scenarios.

Criticality metrics are not only essential for collision avoidance systems but also play a vital role for verification and validation of automated vehicles. With respect to the first application, criticality metrics should be real-time capable and applicable in various traffic situations. For the second application, holistic safety evaluation by criticality metrics is desired. However, existing criticality metrics hardly meet these two requirements. They are either only applicable in post-processing or only assess the safety of maneuvers in longitudinal direction. Therefore, we propose a new acceleration-based criticality metric, which is real-time capable and applicable in both longitudinal and lateral directions. The theory of the proposed criticality metric is introduced and the definition is explained according to different scenarios. A simulation platform is established to validate the criticality metric. The simulation results demonstrate that the proposed criticality metric takes all possible maneuvers into account when meeting a critical situation. Apart from the longitudinal behavior, the lateral behavior of automated vehicles can also be evaluated in real-time. Consequently, it has a wider application scope than other criticality metrics. To demonstrate its contribution to verification and validation of automated vehicles, we apply the criticality metric to a naturalistic driving dataset. The results prove that our criticality metric has a higher precision and recall than Time to Collision. Additionally, it combines the abilities of Time to Collision and Time Head Way to assess the safety of automated vehicles in the longitudinal direction. The proposed criticality metric is real-time capable and is suitable for different situations.