20 December 2023 in IEEE Transactions on Intelligent Transportation Systems
The recent advancements in wireless technology enable connected autonomous vehicles (CAVs) to gather information about their environment by vehicle-to-vehicle (V2V) communication. In this work, we design an information-sharing-based multi-agent reinforcement learning (MARL) framework for CAVs, to take advantage of the extra information when making decisions to improve traffic efficiency and safety. The safe actor-critic algorithm we propose has two new techniques: the truncated $\mathcal{Q}$ -function and safe action mapping. The truncated $\mathcal{Q}$ -function utilizes the shared information from neighboring CAVs such that the joint state and action spaces of the $\mathcal{Q}$ -function do not grow in our algorithm for a large-scale CAV system. We prove the bound of the approximation error between the truncated- $\mathcal{Q}$ and global $Q$ -functions. The safe action mapping provides a provable safety guarantee for both the training and execution based on control barrier functions. Using the CARLA simulator for experiments, we show that our approach improves the CAV system’s efficiency in terms of average velocity and comfort under different CAV ratios and different traffic densities. We also show that our approach avoids the execution of unsafe actions and always maintains a safe distance from other vehicles. We construct an obstacle-at-corner scenario to show that the shared vision can help CAVs to observe obstacles earlier and take action to avoid traffic jams. The experiment video is on https://songyanghan.github.io/cavmarl/.
30 April 2024 in ACM Transactions on Cyber-Physical Systems
Hybrid traffic which involves both autonomous and human-driven vehicles would be the norm of the autonomous vehicles’ practice for a while. On the one hand, unlike autonomous vehicles, human-driven vehicles could exhibit sudden abnormal behaviors such as unpredictably switching to dangerous driving modes—putting its neighboring vehicles under risks; such undesired mode switching could arise from numbers of human driver factors, including fatigue, drunkenness, distraction, aggressiveness, and so on. On the other hand, modern vehicle-to-vehicle (V2V) communication technologies enable the autonomous vehicles to efficiently and reliably share the scarce run-time information with each other [ 1 ]. In this article, we propose, to the best of our knowledge, the first efficient algorithm that can (1) significantly improve trajectory prediction by effectively fusing the run-time information shared by surrounding autonomous vehicles, and can (2) accurately and quickly detect abnormal human driving mode switches or abnormal driving behavior with formal assurance without hurting human drivers’ privacy. To validate our proposed algorithm, we first evaluate our proposed trajectory predictor on NGSIM and Argoverse datasets and show that our proposed predictor outperforms the baseline methods. Then through extensive experiments on SUMO simulator, we show that our proposed algorithm has great detection performance in both highway and urban traffic. The best performance achieves detection rate of 97.3%, average detection delay of 1.2 s, and 0 false alarm.
18 November 2020 in IEEE Journal of Emerging and Selected Topics in Industrial Electronics
Underwater wireless power transfer (UWPT) is a critical infrastructure for supplying power to underwater devices, such as underwater sensors and autonomous underwater vehicles. Enabling a software-defined architecture for UWPT promises to build a flexible and programmable underwater energy network. Although it is crucial that UWPT be made more resilient to cyberattacks and energy stealing, this remains an open challenge. In this article, we propose a software-defined UWPT (SD-UWPT) system that is both cyber and energy secure. A moving target defense approach and an active synchronous detection method are developed to protect the SD-UWPT against scanning-based attacks and power bot attacks. To enable energy-secure UWPT, an impedance measurement based approach is further established to prevent energy stealing. Through comprehensive evaluations, we validate the benefits of SD-UWPT and demonstrate the effectiveness of the proposed cyber–energy secure strategies against various attacks.