17 July 2020 in IEEE Transactions on Smart Grid
This paper develops a model-free volt-VAR optimization (VVO) algorithm via multi-agent deep reinforcement learning (DRL) in unbalanced distribution systems. This method is novel since we cast the VVO problem in distribution networks to an intelligent deep Q-network (DQN) framework, which avoids solving a specific optimization model directly when facing time-varying operating conditions in the systems. We consider statuses/ratios of switchable capacitors, voltage regulators, and smart inverters installed at distributed generators as the action variables of the agents. A delicately designed reward function guides these agents to interact with the distribution system, in the direction of reinforcing voltage regulation and power loss reduction simultaneously. The forward-backward sweep method for radial three-phase distribution systems provides accurate power flow results within a few iterations to the DRL environment. The proposed method realizes the dual goals for VVO. We test this algorithm on the unbalanced IEEE 13-bus and 123-bus systems. Numerical simulations validate the excellent performance of this method in voltage regulation and power loss reduction.
9 June 2020 in IEEE Transactions on Industrial Informatics
Unexpected large power surges will cause instantaneous grid shock, and thus emergency control plans must be implemented to prevent the system from collapsing. In this paper, by the aid of reinforcement learning (RL), novel model-free control (MFC)-based emergency control schemes are presented. Firstly, multi-Q-learning-based emergency plans are designed for limited emergency scenarios by using offline-training-online approximation methods. To solve the more general multi-scenario emergency control problem, a deep deterministic policy gradient (DDPG) algorithm is adopted to learn near-optimal solutions. With the aid of deep Q network, DDPG-based strategies have better generalization abilities for unknown and untrained emergency scenarios, and thus are suitable for multi-scenario learning. Through simulations using benchmark systems, the proposed schemes are proven to achieve satisfactory performances.
30 January 2019 in IEEE Transactions on Smart Grid
The integration of uncertain power resources is causing more challenges for traditional load modeling research. Parameters identification of load modeling is impacted by a variety of load components with time-varying characteristics. This paper develops a deep learning based time-varying parameter identification (TVPI) model for composite load modeling (CLM) with ZIP load and induction motor (IM). A multi-modal long short-term memory (M-LSTM) deep learning method is used to estimate all the time-varying parameters of CLM considering system-wide measurements. It contains a multi-modal structure that makes use of different modalities of the input data to accurately estimate time-varying load parameters. An LSTM network with flexible number of temporal states is defined to capture powerful temporal patterns from the load parameters and measurements time series. The extracted features are further fed to a shared representation layer to capture the joint representation of input time series data. This temporal representation is used in a linear regression model to estimate time-varying load parameters at the current time. Numerical simulations on the 23-and 68-bus systems verify the effectiveness and robustness of the proposed M-LSTM method. Also, the optimal lag values of parameters and measurements as input variables are solved.
11 May 2020 in IEEE Transactions on Industrial Informatics
Battery swapping stations (BSSs) are ideal candidates for fast frequency regulation services (FFRS) due to their large battery stock capacity. In addition, BSSs can precharge batteries for customers and the batteries that are not in charging can provide a stable regulation capacity to the market. However, uncertainties, such as ACE signals and the EV per-hour visit counts, introduce stochastic nonlinear dynamics into the operation of a BSS-based FFRS. Currently, there is no quantification method to ensure its optimal economical operation. To close this gap, in this article, we propose a novel deep Q-learning-based FFRS capacity dynamic scheduling strategy. This method can autonomously schedule the hourly regulation capacity in real time to maximize the BSS's revenue for providing FFRS. Case studies using real-world data verify the efficacy of the proposed work.
17 January 2019 in IEEE Transactions on Smart Grid
Some of the modern smart grid infrastructures, phasor measurement units (PMUs) for instance, are vulnerable to cyberattacks due to their ever-increasing dependence on information and communications technologies. In general, existing solutions to cyberattacks focus on creating redundancy and/or enhancing security levels of sensing and communication networks. These solutions require intensive offline efforts and therefore are economically expensive. Further, they are generally inefficient when dealing with dynamic attacks. This paper proposes a novel density-based spatial clustering approach for online detection, classification, and data recovery for data manipulation attacks to PMU measurements. The proposed method is purely data-driven and is applicable to simultaneous multi-measurement attacks without requiring additional hardware in the existing infrastructure. The proposed approach is also independent of the conventional state estimation. Comprehensive case studies demonstrate the effectiveness of the proposed method.
15 April 2020 in IEEE Transactions on Smart Grid
With the increasing complexity of modern power system, conventional dynamic load modeling with ZIP and induction motors (ZIP + IM) is no longer adequate to address the current load characteristic transitions. In recent years, the Western Electricity Coordinating Council Composite Load Model (WECC CLM) has shown to effectively capture the dynamic load responses over traditional load models in various stability studies and contingency analyses. However, a detailed WECC CLM model typically has a high degree of complexity, with over one hundred parameters, and no systematic approach to identifying and calibrating these parameters. Enabled by the wide deployment of PMUs and advanced deep learning algorithms, proposed here is a double deep Q-learning network (DDQN)-based, two-stage load modeling framework for the WECC CLM. This two-stage method decomposes the complicated WECC CLM for more efficient identification and does not require explicit model details. In the stage one, the DDQN agent determines a proper load composition that can approximate the true transient dynamics. In the second stage, the remaining parameters of the WECC CLM are selected with Monte-Carlo simulations. The proposed method shows that the identified load model is capable of accurately simulating the given dynamics of the reference load model. In addition, the identified load model has strong robustness to represent the reference load model under a wide range of contingencies. The proposed framework is verified using an IEEE 39-bus test system on commercial simulation platforms.