Bi-level optimization and reinforcement learning (RL) constitute the state-of-the-art frameworks for modeling strategic bidding decisions in deregulated electricity markets. However, the former neglects the market participants’ physical non-convex operating characteristics, while conventional RL methods require discretization of state and / or action spaces and thus suffer from the curse of dimensionality. This paper proposes a novel deep reinforcement learning (DRL) based methodology, combining a deep deterministic policy gradient (DDPG) method with a prioritized experience replay (PER) strategy. This approach sets up the problem in multi-dimensional continuous state and action spaces, enabling market participants to receive accurate feedback regarding the impact of their bidding decisions on the market clearing outcome, and devise more profitable bidding decisions by exploiting the entire action domain, also accounting for the effect of non-convex operating characteristics. Case studies demonstrate that the proposed methodology achieves a significantly higher profit than the alternative state-of-the-art methods, and exhibits a more favourable computational performance than benchmark RL methods due to the employment of the PER strategy.

Decarbonization of electricity systems drives significant and continued investments in distributed energy sources to support the cost-effective transition to low-carbon energy systems. However, the rapid integration of distributed photovoltaic (PV) generation presents great challenges in obtaining reliable and secure grid operations because of its limited visibility and intermittent nature. Under this reality, net load forecasting is facing unprecedented difficulty in answering the following question: how can we accurately predict the net load while capturing the massive uncertainties arising from distributed PV generation and load, especially in the context of high PV penetration? This paper proposes a novel probabilistic day-ahead net load forecasting method to capture both epistemic uncertainty and aleatoric uncertainty using Bayesian deep learning, which is a new field that combines Bayesian probability theory and deep learning. The proposed methodological framework employs clustering in subprofiles and considers residential rooftop PV outputs as input features to enhance the performance of aggregated net load forecasting. Numerical experiments have been carried out based on fine-grained smart meter data from the Australian grid with separately recorded measurements of rooftop PV generation and loads. The results demonstrate the superior performance of the proposed scheme compared with a series of state-of-theart methods and indicate the importance and effectiveness of subprofile clustering and high PV visibility.

Multi-energy systems (MES) are attracting increasing attention driven by its potential to offer significant flexibility in future smart grids. At the residential level, the roll-out of smart meters and rapid deployment of smart energy devices call for autonomous multi-energy management systems which can exploit real-time information to optimally schedule the usage of different devices with the aim of minimizing end-users’ energy costs. This paper proposes a novel real-time autonomous energy management strategy for a residential MES using a model-free deep reinforcement learning (DRL) based approach, combining state-of-the-art deep deterministic policy gradient (DDPG) method with an innovative prioritized experience replay strategy. This approach is tailored to align with the nature of the problem by posing it in multi-dimensional continuous state and action spaces, facilitating more cost-effective control strategies to be devised. The superior performance of the proposed approach in reducing end-user’s energy cost while coping with the MES uncertainties is demonstrated by comparing it against state-of-the-art DRL methods as well as conventional stochastic programming and robust optimization methods in numerous case studies in a real-world scenario.

In the last decade, a number of severe urban power outages have been caused by extreme natural disasters, e.g., hurricanes, snowstorms and earthquakes, which highlights the need for rethinking current planning principles of urban energy systems and expanding the classical reliability-oriented view. In addition to being reliable to low-impact and high-probability outages, power system should also have high level of resilience to withstand high-impact and low-probability (HILP) events. Compared with power system, multi-energy systems (MESs) have advantages in improving resilience through energy shifting across multiple energy sectors, a variety of generalized energy storage resources and thermal inertia of heat/cooling loads. This paper proposes a resilience-oriented planning method to determine optimal configuration of distribution level MES, e.g., urban energy supply systems, considering comprehensive impacts from supply, network and demand sides in MES. Impacts of energy shifting at supply side, pipe storage at network side and thermal inertia at demand side are described in the same linear modeling framework using energy hub (EH) model. Generalized energy storage resources including centralized and distributed energy storage devices, pipe network storage and building heat capacity are all modeled into centralized energy storage to facilitate an efficient configuration planning of MES.

The ongoing decarbonisation of modern electricity systems has led to a substantial increase of operational uncertainty, particularly due to the large-scale integration of renewable energy generation. However, the expanding space of possible operating points renders necessary the development of novel security assessment approaches. In this paper we focus on the use of security rules where classifiers are trained offline to characterize previously unseen points as safe or unsafe. This paper proposes a novel deep learning-based feature extraction framework for building security rules. We show how deep autoencoders can be used to transform the space of conventional state variables (e.g., power flows) to a small number of dimensions where we can optimally distinguish between safe and unsafe operation. The proposed framework is data-driven and can be useful in multiple applications within the context of security assessment. To achieve high accuracy, a novel objective-based loss function is proposed to address the issue of imbalanced safe/unsafe classes that characterize electricity system operation. Furthermore, an R-vine copula-based model is proposed to sample historical data and generate large populations of anticipated system states for training. The superior performance of the proposed framework is demonstrated through a series of case studies and comparisons using the load and wind generation data from the French transmission system, which have been mapped to the IEEE 118-bus system.