Reinforcement Learning (RL) has recently impressed the world with stunning results in various applications. While the potential of RL is now well-established, many critical aspects still need to be tackled, including safety and stability issues. These issues, while secondary for the RL community, are central to the control community which has been widely investigating them. Model Predictive Control (MPC) is one of the most successful control techniques because, among others, of its ability to provide such guarantees even for uncertain constrained systems. Since MPC is an optimization-based technique, optimality has also often been claimed. Unfortunately, the performance of MPC is highly dependent on the accuracy of the model used for predictions. In this paper, we propose to combine RL and MPC in order to exploit the advantages of both and, therefore, obtain a controller which is optimal and safe. We illustrate the results with two numerical examples in simulations.

Linear model predictive control (MPC) can be currently deployed at outstanding speeds, thanks to recent progress in algorithms for solving online the underlying structured quadratic programs. In contrast, nonlinear MPC (NMPC) requires the deployment of more elaborate algorithms, which require longer computation times than linear MPC. Nonetheless, computational speeds for NMPC comparable to those of MPC are now regularly reported, provided that the adequate algorithms are used. In this paper, we aim at clarifying the similarities and differences between linear MPC and NMPC. In particular, we focus our analysis on NMPC based on the real-time iteration (RTI) scheme, as this technique has been successfully tested and, in some applications, requires computational times that are only marginally larger than linear MPC. The goal of the paper is to promote the understanding of RTI-based NMPC within the linear MPC community.

Reinforcement Learning (RL) is a powerful tool to perform data-driven optimal control without relying on a model of the system. However, RL struggles to provide hard guarantees on the behavior of the resulting control scheme. In contrast, Nonlinear Model Predictive Control (NMPC) and Economic NMPC (ENMPC) are standard tools for the closedloop optimal control of complex systems with constraints and limitations, and benefit from a rich theory to assess their closedloop behavior. Unfortunately, the performance of (E)NMPC hinges on the quality of the model underlying the control scheme. In this paper, we show that an (E)NMPC scheme can be tuned to deliver the optimal policy of the real system even when using a wrong model. This result also holds for real systems having stochastic dynamics. This entails that ENMPC can be used as a new type of function approximator within RL. Furthermore, we investigate our results in the context of ENMPC and formally connect them to the concept of dissipativity, which is central for the ENMPC stability. Finally, we detail how these results can be used to deploy classic RL tools for tuning (E)NMPC schemes. We apply these tools on both a classical linear MPC setting and a standard nonlinear example from the ENMPC literature.

In this paper, we present a bilevel, model predictive controller for coordination of automated vehicles at intersections. The bilevel controller consists of a coordination level, where intersection occupancy timeslots are allocated, and a vehicle level, where the control commands for the vehicles are computed. We establish persistent feasibility and stability of the bilevel controller under some mild assumptions and derive conditions under which closed-loop collision avoidance can be ensured with bounded position uncertainty. We thereafter detail an implementation of the coordination controller on a three-vehicle test bed, where the intersection-level optimization problem is solved using a distributed Sequential Quadratic Programing method. We present and discuss results from an extensive experimental campaign, where the proposed controller was validated. The experimental results indicate the practical applicability of the bilevel controller and show that safety can be ensured for large positioning uncertainties.

This paper presents a model for day-ahead scheduling of the CHP heat and electric energy production for a residential Microgrid taking into account the economic factors in a liberalized electricity markets, the technical factors in the safety/quality of supply, and the consumer preferences. This day-ahead scheduling model is complemented with a real-time economic MPC model for a subsequent control with respect to the outcomes of the day-ahead scheduling. This combined scheduling and economic MPC provides a general set-up capable of overcoming several major difficulties encountered with a typical scheduling + tracking MPC set-up, e.g. the problems of connecting the economic objectives with different temporal resolution and different requirements in terms of delivery.