Different aggregation levels of the electric grid's big data can be helpful to develop highly accurate deep learning models for Short-term Load Forecasting (STLF) in electrical networks. Whilst different models are proposed for STLF, they are based on small historical datasets and are not scalable to process large amounts of big data as energy consumption data grow exponentially in large electric distribution networks. This paper proposes a novel hybrid clustering-based deep learning approach for STLF at the distribution transformers' level with enhanced scalability. It investigates the gain in training time and the performance in terms of accuracy when clustering-based deep learning modeling is employed for STLF. A k-Medoid based algorithm is employed for clustering whereas the forecasting models are generated for different clusters of load profiles. The clustering of the distribution transformers is based on the similarity in energy consumption profile. This approach reduces the training time since it minimizes the number of models required for many distribution transformers. The developed deep neural network consists of six layers and employs Adam optimization using the TensorFlow framework. The STLF is a day-ahead hourly horizon forecasting. The accuracy of the proposed modeling is tested on a 1,000-transformer substation subset of the Spanish distribution electrical network data containing more than 24 million load records. The results reveal that the proposed model has superior performance when compared to the state-of-the-art STLF methodologies. The proposed approach delivers an improvement of around 44% in training time while maintaining accuracy using single-core processing as compared to non-clustering models.

For the utility to plan the resources accurately and balance the electricity supply and demand, accurate and timely forecasting is required. The proliferation of smart meters in the grids has resulted in an explosion of energy datasets. Processing such data is challenging and usually takes a longer time than the requirement of a short-term load forecast. The paper addresses this concern by utilizing parallel computing capabilities to minimize the execution time while maintaining highly accurate load forecasting models. In this paper, a thousand smart meter energy datasets are analyzed to perform day ahead, hourly short-term load forecast (STLF). The paper utilizes multi-processing to enhance the overall execution time of the forecasting models by submitting simultaneous jobs to all the processors available. The paper demonstrates the efficacy of the proposed approach through the choice of machine learning (ML) models, execution time, and scalability. The proposed approach is validated on real energy consumption data collected at distribution transformers' level in Spanish Electrical Grid. Decision trees have outperformed the other models accomplishing a tradeoff between model accuracy and execution time. The methodology takes only 4 minutes to train 1,000 transformers for an hourly day-ahead forecast of (~24 million records) utilizing 32 processors.

The future of energy systems is currently driven by three key trends: electrification, decarbonization, and digitalization. A growing share of renewable generation is without doubt part of the solution to accommodate the increasing electricity demand with a low-carbon emission constraint. The question addressed in this review paper is to what extent digitalization will enable the decentralization of renewable energy generation close to the point of consumption. Digital technologies, namely, Big Data, Artificial Intelligence, IoT, and Distributed Ledgers, will have a positive impact on renewable Distributed Energy Resources adoption by contributing to achieve a better balance between supply and demand at the edge of the grid and by improving the hosting capacity of distribution networks. However, decentralized energy production is still limited by structural factors (e.g., low power density of renewable resources, economies of scale in renewable generation and storage), regulatory factors (e.g., access to the grid, pricing policy), and digital technology constraints (e.g., limited number of transaction per second). As a result, the likely scenario of development is a reasonable mix between decentralized and (also digital enabled) centralized renewable-based energy generation interconnected together with an engaged customer and highly developed and smarter distribution and transmission networks.

The variability in generation introduced in the electrical system by an increasing share of renewable technologies must be addressed by balancing mechanisms, demand response being a prominent one. In parallel, the massive introduction of smart meters allows for the use of high frequency energy use time series data to segment electricity customers according to their demand response potential. This paper proposes a smart meter time series clustering methodology based on a two-stage k-medoids clustering of normalized load-shape time series organized around the day divided into 48 time points. Time complexity is drastically reduced by first applying the k-medoids on each customer separately, and second on the total set of customer representatives. Further time complexity reduction is achieved using time series representation with low computational needs. Customer segmentation is undertaken with only four easy-to-interpret features: average energy use, energy–temperature correlation, entropy of the load-shape representative vector, and distance to wind generation patterns. This last feature is computed using the dynamic time warping distance between load and expected wind generation shape representative medoids. The two-stage clustering proves to be computationally effective, scalable and performant according to both internal validity metrics, based on average silhouette, and external validation, based on the ground truth embedded in customer surveys.