13 May 2019 in IEEE Access
1 April 2021 in Accident Analysis & Prevention
23 January 2020 in IEEE Access
Urban network traffic congestion can be caused by disturbances, such as fluctuation and disequilibrium of traffic demand. This paper designs a distributed control method for preventing disturbance-based urban network traffic congestion by integrating Multi-Agent Reinforcement Learning (MARL) and regional Mixed Strategy Nash-Equilibrium (MSNE). To enhance the disturbance-rejection performance of Urban Network Traffic Control (UNTC), a regional MSNE concept is integrated, which models the competitive relationship between each agent and its neighboring agents in order to improve the decision-making process of MARL. The learning rate is enhanced with a self-adaptive ability to avoid a local optimal dilemma; Jensen-Shannon (JS) divergence is utilized to define the learning rate of the modified MARL. A two-way rectangular grid network with nine intersections is modeled via a Cell Transmission Model (CTM). A probability distribution mechanism, which can update the turn ratio of each approach dynamically and discretely, is established to represent the segmented route-decision process of the vehicles. The effectiveness of the proposed control method is evaluated through simulations in the grid network. The results show the influence of major disturbances, such as fluctuation of vehicle arrival rate, fluctuation of traffic demand (e.g. a rapidly rising flow and extreme changes in origin-destination distribution), and disequilibrium of traffic demand (e.g. different arrival flows at each boundary of the urban network), on the performance of the suggested control method. The results can be used to improve the state of the art in order to reduce urban network traffic congestion due to these disturbances.
11 April 2025 in Journal of Air Transport Management
20 August 2020 in IEEE Access
The Dynamic Traffic Assignment (DTA) is one of the important measures to alleviate urban network traffic congestion. The congestions are usually caused by stochastic traffic demands, which are generally unassignable from time dimension in the real-world but are assumed to be assignable in existing DTA methods (i.e. real-time travel demands). In this paper, a distributed DTA method for preventing urban network traffic congestion caused by stochastic real-time travel demands by improving Multi-Agent Reinforcement Learning (MARL). A team structure, which consists of decision-makers and advisers, is designed to learn parallelly in realistic DTA tasks. To reduce the size of the solution space adaptively, the dynamic critical values advised by adviser agents are adopted as constraints for the strategy space of decision-makers (i.e. main agents). A collaborative heterogeneous-adviser mechanism is designed to avoid deviation of guidance. To enhance the adaptability of DTA to the changeable external environment, the mixed strategy concept is introduced to improve the decision-making process of main agents. The respective mapping mechanisms are designed to define adaptive learning rates to improve the sensitivity of MARL. The Sioux Falls (SF) network is established as a test platform via a Dynamic Network Loading (DNL). The effectiveness of the suggested DTA method is assessed through numerical simulations SF network. Under the influence of the scenario with stochastic real-time travel demands, the results show that the proposed method outperforms in terms of the throughput of the network and the individual average travel time among the overall network. Additionally, the ability of the proposed method in response to the external environment rapidly has also been demonstrated. Adopting the suggested method can improve the state of the art to assign stochastic real-time travel demands dynamically and to avoid potential traffic congestion fundamentally.
28 May 2023 in Transportation Research Record: Journal of the Transportation Research Board
To improve the efficiency of detecting abnormal traffic incidents on the road network and reduce the false alarm rate, a real-time traffic anomaly detection framework based on a graph spatiotemporal pattern learning (GSTPL) network is proposed. In this framework, a traffic pattern search algorithm based on a fluctuation similarity measure is designed to screen traffic flow data with the same traffic pattern, and a traffic pattern graph tuple is constructed as the input of the network model to avoid the sample imbalance problem and the effect of single-sample randomness for traffic pattern learning. Then the GSTPL network is designed to extract, unsupervised, the traffic spatiotemporal pattern features and make a reasonable prediction of future traffic parameters as the basis for anomaly evaluation. To further restrain the effect of random fluctuations in traffic flow parameters, an abnormal state evaluation method is designed to calculate the anomaly state likelihood by prediction error distribution learning. The overall detection framework realizes stable prediction of network key node traffic parameters by using spatiotemporal pattern features to construct the traffic pattern graph tuple, and gives incident evaluation results in real time by combination with the detection data. The experiment uses I90 and I405 highway traffic data in Seattle, WA, from 2015. Through comparative analysis, the proposed incident detection method based on GSTPL has a higher detection rate and lower false alarm rate, can adaptively learn dynamic changes of the traffic pattern, and has strong adaptability and stability to different traffic environments.