Multi-Agent Reinforcement Learning (MARL) has shown strong advantages in urban multi-intersection traffic signal control, but it also suffers from the problems of non-smooth environment and inter-agent coordination. However, most of the existing research on MARL traffic signal control has focused on designing efficient communication to solve the environment non-smoothness problem, while neglecting the coordination between agents. In order to coordinate among agents, this paper combines MARL and the regional mixed-strategy Nash equilibrium to construct a Deep Convolutional Nash Policy Gradient Traffic Signal Control (DCNPG-TSC) model, which enables agents to perceive the traffic environment in a wider range and achieves effective agent communication and collaboration. Additionally, a Multi-Agent Distributional Nash Policy Gradient (MADNPG) algorithm is proposed in this model, which is the first time the mixed-strategy Nash equilibrium is used for the improvement in the Multi-Agent Deep Deterministic Policy Gradient algorithm traffic signal control strategy to provide the optimal signal phase for each intersection. In addition, the eco-mobility concept is integrated into MARL traffic signal control to reduce pollutant emissions at intersections. Finally, simulation results in synthetic and real-world traffic road networks show that DCNPG-TSC outperforms other state-of-the-art MARL traffic signal control methods in almost all performance metrics, because it can aggregate the information of neighboring agents and optimize the agent’s decisions through gaming to find an optimal joint equilibrium strategy for the traffic road network.
22 December 2023 in Journal of Advanced Transportation
Traffic congestion has been a hot topic of research in the field of intelligent transportation, which can be alleviated by efficient route navigation. Most of the existing route planning methods are non-negotiated algorithms, which do not take into account the route conflicts and collaborative relationships between multiple vehicles. Also, most negotiated algorithms have not been comprehensively considered dynamic route collaboration between vehicles, large-scale efficient computation, environmental pollution, etc. Therefore, an ecological multivehicle real-time route selection model (EMR2SM) for urban road networks is firstly proposed in this paper, which combines real-time traffic conditions of the road network with travel time, distance, and exhaust emissions as optimization indicators. In order to solve the large-scale computation problem of traditional negotiated algorithms, an adaptive multiswarm bee colony (AMSBC) algorithm is designed, which efficiently solves the multivehicle dynamic route selection problem. AMSBC searches the optimal route for each vehicle in parallel through multiple population division and self-adaption mechanism, to make multivehicle route selection reach Nash equilibrium. Compared with three non-negotiated optimization algorithms based on swarm technology, EMR2SM is verified by experiments that it improves the efficiency and accuracy of the optimal route selection for multiple vehicles and reduces vehicle emissions, which can effectively reduce traffic congestion and environmental pollution.
Taxi sharing is a promising method to save resource consumption and alleviate traffic congestion while satisfying people’s commuting needs. Existing research methods include taxi dispatching methods based on intelligent algorithms, single vehicle route recommendation algorithms, and route recommendation algorithms based on vehicle traffic history. However, these studies either focus on how to efficiently dispatch satisfactory vehicles for passengers, ignoring the effect of efficient routes on vehicle travel efficiency, or let vehicles follow the shortest detour distance recommended by the system, ignoring the traffic congestion caused by the influx of vehicles into the same road. To address the abovementioned problems, a noncooperative game for a taxi sharing model (NCG-TSM) in urban road networks is proposed in this paper combined with the traffic conditions of the optional routes, and a distribution estimation algorithm for the shared taxi game is designed to make multivehicle route selections reach Nash equilibrium. The effectiveness of NCG-TSM is verified through simulation experiments. When the number of vehicles reaches the congestion capacity of the road segment, compared to the three common frameworks, the travel time cost and fuel consumption cost can be reduced by 5.8% to 9.1% and 3.5% to 8.9%, respectively. Besides, the occupancy rate has been improved, especially compared to the BMP framework, by 5.5% to 40%.