This paper reviews recent works related to applications of reinforcement learning in power system optimal control problems. Based on an extensive analysis of works in the recent literature, we attempt to better understand the gap between reinforcement learning methods that rely on complete or incomplete information about the model dynamics and data-driven reinforcement learning approaches. More specifically we ask how such models change based on the application or the algorithm, what the currently open theoretical and numerical challenges are in each of the leading applications, and which reinforcement-based control strategies will rise in the following years. The reviewed research works are divided into “model-based” methods and “model-free” methods in order to highlight the current developments and trends within each of these two groups. The optimal control problems reviewed are energy markets, grid stability and control, energy management in buildings, electrical vehicles, and energy storage.
The rapid expansion of Artificial Intelligence is fueling the growth of hyperscale data centers, which introduces significant challenges to existing power systems. This paper aims to provide a comprehensive survey of these integration challenges, specifically from the perspective of power grid utility. We find that AI data centers function as a distinct load category, characterized by high power density, rapid and large-scale power transients, and specific power quality profiles. These attributes create difficulties for long-term resource adequacy and transmission planning due to mismatched development timelines. They also strain real-time grid balancing and introduce risks to system stability, such as voltage and frequency deviations and converter-driven instabilities. The analysis further covers the economic and environmental footprints associated with this new type of consumer. The paper concludes that safely integrating these loads requires a coordinated strategy, encompassing data center-side technologies, grid-enhancing solutions, and new policy frameworks.
With the rapid advancement of deep learning, generative artificial intelligence (Gen-AI) has emerged as a powerful tool, unlocking new prospects in the power systems sector. Despite the evident success of these methods and the rapid growth of this field in the power systems community, there is still a pressing need for a deeper understanding of how different evaluation metrics relate to the underlying statistical structure of the models. Another related important question is what tools can be used to quantify the different uncertainties, which are inherent in these problems, and stem not only from the physical system but also from the nature of the generative model itself. This paper attempts to address these challenges and provides a comprehensive review of existing evaluation metrics for generative models applied in various power system tasks. We analyze how these metrics align with the statistical properties of the models and explore their strengths and limitations. We also examine different sources of uncertainty, distinguishing between uncertainties inherent to the learning model, those arising from measurement errors, and other sources. Our general aim is to promote a better understanding of generative models as they are being applied in power systems to support this fascinating growing trend.
This paper reviews recent works related to applications of reinforcement learning in power system optimal control problems. Based on an extensive analysis of works in the recent literature, we attempt to better understand what is the gap between reinforcement learning methods that rely on complete or incomplete information about the model dynamics, and data-driven reinforcement learning approaches. More specifically we ask how such models change based on the application or the algorithm, what are the currently open theoretical and numerical challenges in each of the leading applications, and which reinforcement-based control strategies will rise in the following years. The reviewed research works are divided to ``model-based'' methods and ``model-free'' methods, in order to highlight the current developments and trends within each of these two groups. The optimal control problems reviewed are energy markets, grid stability and control, energy management in buildings, electrical vehicles and energy storage.
16 September 2025 in Electric Power Systems Research