23 June 2025 in IEEE Transactions on Intelligent Transportation Systems
Ensuring and improving the safety of autonomous driving systems (ADS) is crucial for deployment of highly automated vehicles, especially in safety-critical events. To address the rarity issue, adversarial scenario generation methods are developed, in which behaviors of traffic participants are manipulated to induce safety-critical events. However, existing methods still face two limitations. First, identification of the adversarial participant directly impacts the effectiveness of the generation. However, complexity of real-world scenarios, with numerous participants and diverse behaviors, makes identification challenging. Second, potential of generated safety-critical scenarios to continuously improve ADS performance remains underexplored. To address these issues, we propose LLM-attacker: a closed-loop adversarial scenario generation framework leveraging large language models (LLMs). Specifically, multiple LLM agents are designed and coordinated to identify optimal attackers. Then, the trajectories of attackers are optimized to generate adversarial scenarios. These scenarios are iteratively refined based on the performance of ADS, forming a feedback loop to improve ADS. Experimental results show that LLM-attacker can create more dangerous scenarios than other methods, and the ADS trained with it achieves a collision rate half that of training with normal scenarios. This indicates the ability of LLM-attacker to test and enhance the safety and robustness of ADS. The framework’s closed-loop design enables continuous scenario evolution compliant with regulatory standards, supporting both safety assurance and policy verification for ADS. Video demonstrations are provided at: https://drive.google.com/file/d/15rROV_ 8LUcc2jXSuSNBHVOHMCFKSn__B/view
14 May 2024 in IEEE Transactions on Intelligent Vehicles
Highly Automated Vehicles (HAVs) are exposed to numerous unexpected faults that threaten the functionality of the Autonomous Driving System (ADS) in HAVs, and even minor faults can lead to serious consequences such as collisions. Accordingly, fault tolerance of HAVs should be thoroughly evaluated before large-scale deployment. Fault Injection (FI) testing is commonly used for the verification and validation of HAVs. However, due to the time cost of FI simulation, it is impossible to simulate all combinations of initial conditions and the high-dimensional fault space. Meanwhile, the inherent uncertainty in complex ADS of the HAV under test cause uncertain testing results, which leads to unreliable results in one-time simulation. To address these problems, an accelerated FI method considering uncertainty within ADS based on Dynamic Bayesian Network (DBN) is proposed. DBN is applied to serve as a surrogate for HAVs and to learn the causal relationship between factors in the complex system of HAVs. Rolling Forecast and Monte Carlo sampling are combined to predict the collision probability after FI. Taking open-source Autonomous Driving System Baidu Apollo as the System Under Test, the experimental results demonstrate that the DBN-based FI method performs well both in efficiency and accuracy across various scenarios. DBN-based FI is 447 times faster than simulation and can achieve 88.9% precision. Furthermore, the collision probability and the variable distribution calculated by uncertain prediction are close to those obtained by simulation.