Model-based and data-driven methods are commonly used in leak location strategies in water distribution networks. This paper formulates a hybrid methodology in two stages that complements the advantages and disadvantages of data-driven and model-based strategies. In the first stage, a support vector machine multiclass classifier is used to reduce the search space for the leak location task. In the second stage, leak location task is formulated as an inverse problem, and solved using a variation of the differential evolution algorithm called topological differential evolution. The robustness of the method is tested considering measurement and varying demand uncertainty conditions ranging from 5 to 15% of node nominal demands. The performance of the hybrid method is compared to the support vector machine classifier and topological differential evolution approaches as standalone methods of leak location. The hybrid proposal shows higher performance in terms of location accuracy, zone size, and computational load.

Variable Speed Limit (VSL) is a promising highway traffic management strategy deployed worldwide. Most algorithms deployed operate based on a set of predefined rules which hinders their ability to perform optimally across diverse traffic conditions. Recent studies applying multi-agent reinforcement learning (MARL) techniques to VSL control problems have shown promising results. However, none of the existing MARL frameworks comply with real-world requirements imposed by most traffic agencies across the United States. In this work, we propose MARVEL (Multi-Agent Reinforcement-learning for large-scale Variable spEed Limits), a novel MARL framework for large-scale VSL highway control with real-world deployment capabilities. MARVEL only relies on sensing information available in most real-world settings and learns through a reward structure that incorporates three abstract goals to ensure reasonable behaviors under different traffic conditions. Through parameter sharing among all VSL agents, the proposed framework scales to cover corridors equipped with multiple VSL gantries. The policies are trained in a microscopic traffic simulation environment, focusing on a short freeway stretch with 8 VSL agents spanning 7 miles. For testing, these policies are applied to a more extensive network with 34 VSL agents spanning 17 miles of Interstate 24 (I-24) near Nashville, Tennessee, the United States. MARVEL improves traffic safety by 63.4% compared to the no-control scenario and enhances traffic mobility by 58.6% compared to a state-of-the-practice algorithm that has been deployed on I-24. Finally, we test the response of the policy learned in simulation with real-world data collected from I-24 and illustrate its deployment capability.

Fault tolerant control (FTC) focuses on developing algorithms to accommodate the impact of system faults while allowing the system to continuously operate in a degraded manner. Additionally, data-driven methods like reinforcement learning (RL) have shown excellent performance for complex continuous control tasks. However, faults affect the system dynamics which disrupt optimal policy guarantees as the Markov Property cannot be satisfied. In this work, we propose a scheme based on a combination of parameter estimation, RL, and model-based control to handle faults in a continuous control environment. We empirically demonstrate our approach on a complex octocopter trajectory-tracking task subject to single and multi motor faults. We show improved performance compared with nominal hierarchical PID control for large magnitude faults. Lastly, we demonstrate our approach’s robustness against noisy parameter estimation.

This article presents the first field deployment of a multiagent reinforcement learning (MARL)-based variable- speed-limit (VSL) control system on Interstate 24 (I-24) near Nashville, TN, USA. We design and demonstrate a full pipeline from training MARL agents in a traffic simulator to a field deployment on a 17-mi segment of I-24 encompassing 67 VSL controllers. The system was launched on 8 March 2024 and has made approximately 35 million decisions on 28 million trips in six months of operation. We apply an invalid action masking mechanism and several safety guards to ensure real-world constraints. The MARL-based implementation operates up to 98% of the time, with the safety guards overriding the MARL decisions for the remaining time. We evaluate the performance of the MARL-based algorithm in comparison to a previously deployed non-RL VSL benchmark algorithm on I-24. The results show that the MARL-based VSL control system achieves a superior performance.