The development of autonomous vehicles has shown great potential to enhance the efficiency and safety of transportation systems. However, the decision-making issue in complex human-machine mixed traffic scenarios, such as unsignalized intersections, remains a challenge for autonomous vehicles. While reinforcement learning (RL) has been used to solve complex decision-making problems, existing RL methods still have limitations in dealing with cooperative decision-making of multiple connected autonomous vehicles (CAVs), ensuring safety during exploration, and simulating realistic human driver behaviors. In this paper, a novel and efficient algorithm, Multi-Agent Game-prior Attention Deep Deterministic Policy Gradient (MA-GA-DDPG), is proposed to address these limitations. Our proposed algorithm formulates the decision-making problem of CAVs at unsignalized intersections as a decentralized multi-agent reinforcement learning problem and incorporates an attention mechanism to capture interaction dependencies between ego CAV and other agents. The attention weights between the ego vehicle and other agents are then used to screen interaction objects and obtain prior hierarchical game relations, based on which a safety inspector module is designed to improve the traffic safety. Furthermore, both simulation and hardware-in-the-loop experiments were conducted, demonstrating that our method outperforms other baseline approaches in terms of driving safety, efficiency, and comfort.

Connected Autonomous Vehicles (CAVs) are being tested globally, but their performance in complex scenarios remains suboptimal. While cooperative driving improves CAV performance by leveraging vehicle collaboration, its lack of interaction and continuous learning limits current applications to single scenarios and specific Cooperative Driving Automation (CDA). To address these issues, this paper proposes CoDrivingLLM, an interactive and learnable LLM-driven cooperative driving framework for all-scenario and all-CDA applications. First, an environment module updates vehicle positions based on semantic decisions, mitigating errors from LLM-controlled positioning. Second, leveraging the four CDA levels defined in SAE J3216, a centralized-distributed coupled architecture reasoning module is developed to ensure safe and efficient cooperation through centralized negotiation and distributed decision. Finally, by introducing a memory module that employs Retrieval Augmented Generation (RAG), CAVs are endowed with the ability to learn from their past experiences to avoid repeating mistakes. Through ablation studies and comparisons with other cooperative driving methods, the results demonstrate that the proposed CoDrivingLLM significantly enhances safety, efficiency, and adaptability across various scenarios. Our code is available at: https://github.com/FanGShiYuu/CoDrivingLLM.

The advent of autonomous vehicles (AVs) alongside human-driven vehicles (HVs) has ushered in an era of mixed traffic flow, presenting a significant challenge: the intricate interaction between these entities within complex driving environments. AVs are expected to have human-like driving behavior to seamlessly integrate into human-dominated traffic systems. To address this issue, we propose a reinforcement learning framework that considers driving priors and Social Coordination Awareness (SCA) to optimize the behavior of AVs. The framework integrates a driving prior learning (DPL) model based on a variational autoencoder to infer the driver's driving priors from human drivers' trajectories. A policy network based on a multi-head attention mechanism is designed to effectively capture the interactive dependencies between AVs and other traffic participants to improve decision-making quality. The introduction of SCA into the autonomous driving decision-making system, and the use of Coordination Tendency (CT) to quantify the willingness of AVs to coordinate the traffic system is explored. The unsignalized intersection serves as a representative experimental scenario. Simulation results show that the proposed framework can not only improve the decision-making quality of AVs but also motivate them to produce social behaviors, with potential benefits for the safety and traffic efficiency of the entire transportation system.

The integration of Autonomous Vehicles (AVs) into existing human-driven traffic systems poses considerable challenges, especially within environments where human and machine interactions are frequent and complex, such as at unsignalized intersections. To deal with these challenges, we introduce a novel framework predicated on dynamic and socially-aware decision-making game theory to augment the social decision-making prowess of AVs in mixed driving environments. This comprehensive framework is delineated into three primary modules: Interaction Orientation Identification, Mixed-Strategy Game Modeling, and Expert Mode Learning. We introduce ’Interaction Orientation’ as a metric to evaluate the social decision-making tendencies of various agents, incorporating both environmental factors and trajectory characteristics. The mixed-strategy game model developed as part of this framework considers the evolution of future traffic scenarios and includes a utility function that balances safety, operational efficiency, and the unpredictability of environmental conditions. To adapt to real-world driving complexities, our framework utilizes a dynamic optimization framework for assimilating and learning from expert human driving strategies. These strategies are compiled into a comprehensive strategy library, serving as a reference for future decision-making processes. The proposed approach is validated through extensive driving datasets and human-in-loop driving experiments, and the results demonstrate marked enhancements in decision timing and precision.

With the aim of promoting energy conservation and emission reduction, electric vehicles (EVs) have gained significant attention as a strategic industry in many countries. However, the insufficiency of accessible charging infrastructure remains a challenge, hindering the widespread adoption of EVs. To address this issue, we propose a novel approach to optimize the placement of charging stations within a road network, known as the charging station location problem (CSLP). Our method considers multiple factors, including fairness in charging station distribution, benefits associated with their placement, and drivers’ discomfort. Fairness is quantified by the balance in charging station coverage across the network, while driver comfort is measured by the total time spent during the charging process. Then, the CSLP is formulated as a reinforcement learning problem, and we introduce a novel model called PPO-Attention. This model incorporates an attention layer into the Proximal Policy Optimization (PPO) algorithm, enhancing the algorithm’s capacity to identify and understand the intricate interdependencies between different nodes in the network. We have conducted extensive tests on urban road networks in Europe, North America, and Asia. The results demonstrate the superior performance of our approach compared to existing baseline algorithms. On average, our method achieves a profit increase of 258.04% and reduces waiting time by 73.40%, travel time by 18.46%, and charging time by 40.10%.

The cooperative driving technology of Connected and Autonomous Vehicles (CAVs) is crucial for improving the efficiency and safety of transportation systems. Learning-based methods, such as Multi-Agent Reinforcement Learning (MARL), have demonstrated strong capabilities in cooperative decision-making tasks. However, existing MARL approaches still face challenges in terms of learning efficiency and performance. In recent years, Large Language Models (LLMs) have rapidly advanced and shown remarkable abilities in various sequential decision-making tasks. To enhance the learning capabilities of cooperative agents while ensuring decision-making efficiency and cost-effectiveness, we propose LDPD, a language-driven policy distillation method for guiding MARL exploration. In this framework, a teacher agent based on LLM trains smaller student agents to achieve cooperative decision-making through its own decision-making demonstrations. The teacher agent enhances the observation information of CAVs and utilizes LLMs to perform complex cooperative decision-making reasoning, which also leverages carefully designed decision-making tools to achieve expert-level decisions, providing high-quality teaching experiences. The student agent then refines the teacher's prior knowledge into its own model through gradient policy updates. The experiments demonstrate that the students can rapidly improve their capabilities with minimal guidance from the teacher and eventually surpass the teacher's performance. Extensive experiments show that our approach demonstrates better performance and learning efficiency compared to baseline methods.