8 April 2024 in ACM Journal on Autonomous Transportation Systems
This work investigates traffic control via controlled connected and autonomous vehicles (CAVs) using novel controllers derived from the linear-quadratic regulator (LQR) theory. CAV-platoons are modeled as moving bottlenecks impacting the surrounding traffic with their speeds as control inputs. An iterative controller algorithm based on the LQR theory is proposed along with a variant that allows for penalizing abrupt changes in platoon speeds. The controllers use the Lighthill-Whitham-Richards (LWR) model implemented using an extended cell transmission model (CTM), which considers the capacity drop phenomenon for a realistic representation of traffic in congestion. The impact of various parameters of the proposed controller on the control performance is analyzed. The effectiveness of the proposed traffic control algorithms is tested using a traffic control example and compared with existing proportional-integral (PI) and model predictive control (MPC) controllers from the literature. A case study using the TransModeler traffic microsimulation software is conducted to test the usability of the proposed controller as well as existing controllers in a realistic setting and derive qualitative insights. It is observed that the proposed controller works well in both settings to mitigate the impact of the jam caused by a fixed bottleneck. The computation time required by the controller is also small, making it suitable for real-time control.
31 October 2024 in IEEE Access
Variable Speed Limit (VSL) is a promising highway traffic management strategy deployed worldwide. Most algorithms deployed operate based on a set of predefined rules which hinders their ability to perform optimally across diverse traffic conditions. Recent studies applying multi-agent reinforcement learning (MARL) techniques to VSL control problems have shown promising results. However, none of the existing MARL frameworks comply with real-world requirements imposed by most traffic agencies across the United States. In this work, we propose MARVEL (Multi-Agent Reinforcement-learning for large-scale Variable spEed Limits), a novel MARL framework for large-scale VSL highway control with real-world deployment capabilities. MARVEL only relies on sensing information available in most real-world settings and learns through a reward structure that incorporates three abstract goals to ensure reasonable behaviors under different traffic conditions. Through parameter sharing among all VSL agents, the proposed framework scales to cover corridors equipped with multiple VSL gantries. The policies are trained in a microscopic traffic simulation environment, focusing on a short freeway stretch with 8 VSL agents spanning 7 miles. For testing, these policies are applied to a more extensive network with 34 VSL agents spanning 17 miles of Interstate 24 (I-24) near Nashville, Tennessee, the United States. MARVEL improves traffic safety by 63.4% compared to the no-control scenario and enhances traffic mobility by 58.6% compared to a state-of-the-practice algorithm that has been deployed on I-24. Finally, we test the response of the policy learned in simulation with real-world data collected from I-24 and illustrate its deployment capability.
14 August 2025 in IEEE Intelligent Transportation Systems Magazine
This article presents the first field deployment of a multiagent reinforcement learning (MARL)-based variable- speed-limit (VSL) control system on Interstate 24 (I-24) near Nashville, TN, USA. We design and demonstrate a full pipeline from training MARL agents in a traffic simulator to a field deployment on a 17-mi segment of I-24 encompassing 67 VSL controllers. The system was launched on 8 March 2024 and has made approximately 35 million decisions on 28 million trips in six months of operation. We apply an invalid action masking mechanism and several safety guards to ensure real-world constraints. The MARL-based implementation operates up to 98% of the time, with the safety guards overriding the MARL decisions for the remaining time. We evaluate the performance of the MARL-based algorithm in comparison to a previously deployed non-RL VSL benchmark algorithm on I-24. The results show that the MARL-based VSL control system achieves a superior performance.