Model predictive control (MPC) and deep reinforcement learning (DRL) have been developed extensively as two independent techniques for traffic management. Although the features of MPC and DRL complement each other very well, few of the current studies consider combining these two methods for application in the field of freeway traffic control. This paper proposes a novel framework for integrating MPC and DRL methods for freeway traffic control that is different from existing MPC-(D)RL methods. Specifically, the proposed framework adopts a hierarchical structure, where a high-level efficient MPC component works at a low frequency to provide a baseline control input, while the DRL component works at a high frequency to modify online the output generated by MPC. The control framework, therefore, needs only limited online computational resources and is able to handle uncertainties and external disturbances after proper learning with enough training data. The proposed framework is implemented on a benchmark freeway network in order to coordinate ramp metering and variable speed limits, and the performance is compared with standard MPC and DRL approaches. The simulation results show that the proposed framework outperforms standalone MPC and DRL methods in terms of total time spent (TTS) and constraint satisfaction, despite model uncertainties and external disturbances.

Distributed model predictive control (DMPC) has become a focus in the energy management of shipboard power systems due to its capabilities for privacy preservation, robustness, and distributing computing burdens to local processors. DMPC determines control actions in a distributed manner based on the predictions of system statuses. However, the performance of DMPC is affected by inaccurate predictions resulting from uncertain parameters in nominal prediction models. Particularly, these inaccuracies in predicting propulsion loads and solar panel generation powers can lead to power imbalances when implementing the control actions determined by DMPC. To address this challenge, this paper proposed a novel reinforcement learning compensated DMPC (RL-C-DMPC) to distributively compensate for the control actions determined by DMPC baseline control, thereby rectifying the power imbalances caused by uncertain parameters in nominal prediction models. A value-decomposition-network-based training and distributed testing mechanism is designed for our proposed RL-C-DMPC. Furthermore, a method for range selection of compensation rate is specifically proposed for the energy management of shipboard power systems. To validate the effectiveness of our proposed RL-C-DMPC, we conduct a comprehensive case study utilizing real-life voyage data and historical solar power generation data in the area of the voyage to build the environment for training and testing. By comparing power imbalances between DMPC and RL-C-DMPC, our results indicate significant reductions in power imbalances so that frequency stability can be better ensured. Furthermore, via the case study, we also evaluate the communication robustness of RL-C-DMPC.