10 July 2025 in Proceedings of the IEEE
Given the availability of more comprehensive measurement data in modern power systems, reinforcement learning (RL) has gained significant interest in operation and control. Conventional RL relies on trial-and-error interactions with the environment and reward feedback, which often leads to exploring unsafe operating regions and executing unsafe actions, especially when deployed in real-world power systems. To address these challenges, safe RL has been proposed to optimize operational objectives while ensuring safety constraints are met, keeping actions and states within safe regions throughout both training and deployment. Rather than relying solely on manually designed penalty terms for unsafe actions, as is common in conventional RL, safe RL methods reviewed here primarily leverage advanced and proactive mechanisms. These include techniques such as Lagrangian relaxation, safety layers, and theoretical guarantees like Lyapunov functions to rigorously enforce safety boundaries. This article provides a comprehensive review of safe RL methods and their applications across various power system operations and control domains, including security control, real-time operation, operational planning, and emerging areas. It summarizes existing safe RL techniques, evaluates their performance, analyzes suitable deployment scenarios, and examines algorithm benchmarks and application environments. This article also highlights real-world implementation cases and identifies critical challenges such as scalability in large-scale systems and robustness under uncertainty, providing potential solutions and outlining future directions to advance the reliable integration and deployment of safe RL in modern power systems.
1 November 2023 in IEEE Transactions on Power Systems
13 April 2021 in IEEE Transactions on Industrial Informatics
The widely used transient stability-constrained optimal power flow (TSC-OPF) method for power system preventive control is very time-consuming and thus not applicable for large-scale systems. This article proposes a new deep learning-enabled surrogate model that can significantly improve computational efficiency while maintaining high accuracy. To achieve that, the deep belief network (DBN) is strategically integrated with the reference-point-based nondominated sorting genetic algorithm (NSGA-III) to develop a new preventive control framework. The DBN allows us to identify the mapping relationship between the transient stability index and system operational features. The identified functional mapping relationship is further used as the surrogate to connect the DBN results with TSC-OPF for preventive control. The integrated NSGA-III and surrogate model enable the multiobjective optimization to consider various constraints and objectives, such as minimization of costs of generation dispatch cost and load shedding while maintaining the system stability. Extensive simulation results on several IEEE test systems show that the proposed method can achieve highly efficient control solutions and outperform other alternatives in terms of computational efficiency and economic benefits.
9 December 2020 in IEEE Transactions on Power Systems
To address the uncertainties of renewable energy and loads in transient stability assessment with credible contingencies, this letter proposes a stacked denoising autoencoder (SDAE)-based probabilistic prediction method. The correlations among wind farms have been effectively considered through the variable transformation via the Cholesky decomposition. SDAE allows us to learn the mapping relationship between operational features and the transient stability margin. The possible operation scenarios are sampled under different confidence levels to generate appropriate inputs for SDAE to assess the probabilistic transient stability distribution. Results on the modified IEEE 39-bus system show that our proposed method can achieve a similar level of accuracy as benchmark and improved Monte Carlo simulations-based methods while having much higher computational efficiency.
15 January 2025 in Nature Reviews Clean Technology
Renewable energy source integration into energy systems can contribute to transmission congestion, which requires time-consuming and capital-intensive upgrades to address. Grid-enhancing technologies (GETs) can increase the capacity of grids with minimal investment, preventing congestion and curtailment of renewable energy. In this Review, we discuss the principles and uses of GETs, which use software and/or hardware to interpret real-time conditions to better use the existing capacity of grid assets. GETs include dynamic line ratings, dynamic transformer ratings, power flow controls, topology optimization, advanced conductor technologies, energy storage systems, and demand response. These GETs can enhance system performance individually, but the deployment of multiple GETs together would greatly increase their effect on the grid capacity and stability by removing multiple capacity bottlenecks in parallel. Infrastructure for real-time data acquisition, transmission and analysis is key to successfully deploying GETs but requires further development and commercialization for broader deployment.
27 April 2023 in IEEE Transactions on Power Systems
This paper proposes a deep sigma point processes (DSPP)-assisted chance-constrained power system transient stability preventive control method to deal with uncertain renewable energy and loads-induced stability risk. The traditional transient stability-constrained preventive control is reformulated as a chance-constrained optimization problem. To deal with the computational bottleneck of the time-domain simulation-based probabilistic transient stability assessment, the DSPP is developed. DSPP is a parametric Bayesian approach that allows us to predict system transient stability with high computational efficiency while accurately quantifying the confidence intervals of the predictions that can be used to inform system instability risk. To this end, with a given preset confidence probability, we embed DSPP into the primal dual interior point method to help solve the chance-constrained preventive control problem, where the corresponding Jacobian and Hessian matrices are derived. Comparison results with other existing methods show that the proposed method can significantly speed up preventive control while maintaining high accuracy and convergence.