In this letter, a novel autonomous control framework “Grid Mind” is proposed for the secure operation of power grids based on cutting-edge artificial intelligence (AI) technologies. The proposed platform provides a data-driven, model-free and closed-loop control agent trained using deep reinforcement learning (DRL) algorithms by interacting with massive simulations and/or real environment of a power grid. The proposed agent learns from scratch to master the power grid voltage control problem purely from data. It can make autonomous voltage control (AVC) strategies to support grid operators in making effective and timely control actions, according to the current system conditions detected by real-time measurements from supervisory control and data acquisition (SCADA) or phasor measurement units (PMUs). Two state-of-the-art DRL algorithms, namely deep Q-network (DQN) and deep deterministic policy gradient (DDPG), are proposed to formulate the AVC problem with performance compared. Case studies on a realistic 200-bus test system demonstrate the effectiveness and promising performance of the proposed framework.

The complexity of modern power grids keeps increasing due to the expansion of renewable energy resources and the requirement of fast demand responses, which results in a great challenge for conventional power grid control systems. Existing autonomous control approaches for the power grid requires an accurate system model and a powerful computational platform, which is difficult to scale up for the large-scale energy system with more control options and operating conditions. Facing these challenges, this paper proposes a data-driven multi-agent power grid control scheme using a deep reinforcement learning (DRL) method. Specifically, the classic autonomous voltage control (AVC) problem is taken as an example and formulated as a Markov Game with a heuristic method to partition agents. Then, a multi-agent AVC (MA-AVC) algorithm based on a multi-agent deep deterministic policy gradient (MADDPG) method that features centralized training and decentralized execution is developed to solve the AVC problem. The proposed method can learn from scratch and gradually master the system operation rules by input and output data. In order to demonstrate the effectiveness of the proposed MA-AVC algorithm, comprehensive case studies are conducted on an Illinois 200-Bus system considering load/generation changes, N-1 contingencies, and weak centralized communication environment.

With the rapid growth of electric vehicle (EV) ownership, attentions have been paid to the foundation of EVs, the electric vehicle supply equipment (EVSE). Different approaches of effort, among which battery swapping and fast charging are the two most well studied, have been made to solve the tradeoff problem between the battery charging speed and battery lifetime. There has been considerable debate over development strategy between charging and battery swapping. In passenger vehicles, the EV charging mode seems to dominate. But, does it mean that the battery swap mode is a dead-end? The answer should be “No”. There are use cases showing that battery swap can have great potentials for some particular uses, such as taxis and buses. This paper uses Monte Carlo simulations of vehicle behaviors to compare the service capacities and earnings of EV charging and battery swapping for both taxi and bus fleets. Stochastic models of taxis, buses, charging stations (CSs) and battery swapping systems are set up. Subsequently, service capacities of the EVSE are compared. The impact of factors on the service capacity, such as the size of the vehicle's battery, vehicle's moving speed, the power of the CS, and the price of the swapping service is investigated. Finally, possible reasons of today's less prevalence of battery swapping stations are discussed. The results of the analysis, which can be helpful to policymakers and industry investors, show that with same service capacity, an EV battery swapping station could provide significantly more financial and social benefits for the vehicle operators and EVSE service providers than that of an EV CS.

In this paper, a reinforcement-learning-based online optimal (RL-OPT) control method is proposed for hybrid energy storage system (HESS) in AC/DC microgrids involving photovoltaic (PV) systems and diesel generators (DG). Due to the low system inertia, conventional unregulated charging and discharging (C&D) of energy storages in microgrids may introduce disturbances that degrade the power quality and system performance, especially in fast C&D situations. Secondary and tertiary control levels can optimize the state of charge (SOC) reference of HESS; however, they are lacking the direct controllability of regulating the transient performance. Additionally, the uncertainties in practical systems greatly limit the performance of conventional model-based controllers. In this study, the optimal control theory is applied to optimize the C&D profile and to suppress the disturbances caused by integrating HESS. Neural networks (NN) are devised to estimate the nonlinear dynamics of HESS based on the input/output measurements, and to learn the optimal control input for bidirectional-converter-interfaced HESS using the estimated system dynamics. Because the proposed RLOPT method is fully decentralized, which only requires the local measurements, the plug & play capability of HESS can be easily realized. Both islanded and grid-tied modes are considered. Extensive simulations and experiments are conducted to evaluate the effectiveness of the proposed method.

Existing power system resilience enhancement methods, such as proactive generation rescheduling, movable sources dispatch, and network topology reconfiguration, do not explore the capability and flexibility of shunts to maintain voltage stability during and after disrupting events. Besides, existing methods rely on accurate system models that are not easily scalable for large integrated power grids. In this paper, a data-driven multi-agent framework based on a deep-reinforcement-learning algorithm is proposed to overcome the computation and scalability concerns related to precise system models and to plan for the deployment of shunts for power system resilience enhancement. Specifically, voltage violations due to outages of multiple lines during windstorms are taken as an example of a power system resilience improvement problem. Then, a multi-agent based hybrid soft actor critic (HSAC) algorithm is developed for offline siting and sizing as well as online controlling of shunt reactive power compensators to enhance voltage resilience. The HSAC algorithm is derived from the fundamental SAC algorithms that contain both continuous and discrete action spaces. The proposed multi-agent framework learns from previous experiences and eventually gets trained to determine proper locations and sizes for shunts to avoid voltage violations during multiple line failures. The proposed approach is demonstrated on the IEEE 57-bus and IEEE 300-bus systems. The results show that the proposed multi-agent framework is effective for installation planning and controlling of shunts to enhance power system resilience.

The concept of peer-to-peer (P2P) trading, or transactive energy (TE), is gaining momentum as a future grid restructure. It has the potentials to utilize distributed energy resources (DERs), proactive demand side management (DSM), and the infusion in information and communication technologies (e.g., blockchain and internet of things (IoT)) for promoting the technical and economic efficiency of the system in its entirety. An efficient market framework is vital for the successful and sustainable implementation of such a concept. This paper proposes a P2P energy trading framework enabled by blockchain. It consolidates bilateral contracts, an electronic-commerce platform, a double-auction Vickrey-Clarke-Groves (VCG) mechanism, and trading functionalities with the main grid. Through these multi-layer mechanisms, various trading preferences and attributes of electricity generation and/or consumption are accommodated. Meanwhile, the VCG mechanism eliminates any potential for market power exercise via incentivizing truthful bidding of participants. Different remedies are proposed to overcome the drawback of VCG, i.e., the lack of balanced-budget property. Accordingly, the proposed trading framework is described as multi-settlement and quasi-ideal. Case studies are conducted to analyze and evaluate the proposed trading framework and demonstrate the effectiveness of the proposed remedies in handling probable market deficiencies.