Smart grids have been gradually replacing the traditional power grids since the last decade. Such transformation is linked to adding a large number of smart meters and other sources of information extraction units. This provides various theopportunities of havingassociated with the collected big data.Now, Hence, the triumph of the smart grid utilizationenergy paradigm depends on the factor of smart gridbig data analytics. which wouldThis includes the effective acquisition, transmission, processing,and effective visualization, interpretation, and utilization of big dataof the data that would bring benefits and increase efficiency of the power systems. The paper provides deep insights aboutinto various big data technologies and discusses the literature onbig data analytics in the context of the smart grid. Also, itThe paper also presents the challenges and opportunities , tools and emerging technologiesbrought by the advent ofthe internet of things, machine learning and big data from smart gridsand cloud platforms.

Different aggregation levels of the electric grid's big data can be helpful to develop highly accurate deep learning models for Short-term Load Forecasting (STLF) in electrical networks. Whilst different models are proposed for STLF, they are based on small historical datasets and are not scalable to process large amounts of big data as energy consumption data grow exponentially in large electric distribution networks. This paper proposes a novel hybrid clustering-based deep learning approach for STLF at the distribution transformers' level with enhanced scalability. It investigates the gain in training time and the performance in terms of accuracy when clustering-based deep learning modeling is employed for STLF. A k-Medoid based algorithm is employed for clustering whereas the forecasting models are generated for different clusters of load profiles. The clustering of the distribution transformers is based on the similarity in energy consumption profile. This approach reduces the training time since it minimizes the number of models required for many distribution transformers. The developed deep neural network consists of six layers and employs Adam optimization using the TensorFlow framework. The STLF is a day-ahead hourly horizon forecasting. The accuracy of the proposed modeling is tested on a 1,000-transformer substation subset of the Spanish distribution electrical network data containing more than 24 million load records. The results reveal that the proposed model has superior performance when compared to the state-of-the-art STLF methodologies. The proposed approach delivers an improvement of around 44% in training time while maintaining accuracy using single-core processing as compared to non-clustering models.

Forecasting of energy consumption in Smart Buildings (SB) and using the extracted information to plan and operate power generation are crucial elements of the Smart Grid (SG) energy management. Prediction of electrical loads and scheduling the generation resources to match the demand enable the utility to mitigate the energy generation cost. Different methodologies have been employed to predict energy consumption at different levels of distribution and transmission systems. In this paper, a novel hybrid deep learning model is proposed to predict energy consumption in smart buildings. The proposed framework consists of two stages, namely, data cleaning, and model building. The data cleaning phase applies pre-processing techniques to the raw data and adds additional features of lag values. In the model-building phase, the hybrid model is trained on the processed data. The hybrid deep learning (DL) model is based on the stacking of fully connected layers, and unidirectional Long Short Term Memory (LSTMs) on bi-directional LSTMs. The proposed model is designed to capture the temporal dependencies of energy consumption on dependent features and to be effective in terms of computational complexity, training time, and forecasting accuracy. The proposed model is evaluated on two benchmark energy consumption datasets yielding superior performance in terms of accuracy when compared with widely used hybrid models such as Convolutional (Conv) Neural Network-LSTM, ConvLSTM, LSTM encoder-decoder model, stacking models, etc. A mean absolute percentage error (MAPE) of 2.00% for case study 1 and a MAPE of 3.71% for case study 2 is obtained for the proposed forecasting DL model in comparison with LSTM-based models that yielded 7.80% MAPE and 5.099% MAPE for two datasets respectively. The proposed model has also been applied for multi-step week-ahead daily forecasting with an improvement of 8.368% and 20.99% in MAPE against the LSTM-based model for the utilized energy consumption datasets respectively.

Smart grids are re-engineering the electricity transmission and distribution system throughout the world. It is an amalgam of increased digital information with the electrical power grids. Managing the data generated from the grid efficiently is the key to successful knowledge extraction from the smart grid big data. Most of the scientific advancements are becoming data-driven and becoming an interesting area of research for data scientists. It is challenging the world computationally enough to develop new storage methods and data processing technologies. Managing big data involves data cleaning, integration of varied data sources, and decision-making applications. This paper focuses on the study of big data management and proposes a management process to help manage the data in the grid. Data management tools and techniques have been leveraged in understanding the sources and data types in the grid. The paper emphasizes the limitations of the existing solutions inclined towards applications of the smart grid big data.

For the utility to plan the resources accurately and balance the electricity supply and demand, accurate and timely forecasting is required. The proliferation of smart meters in the grids has resulted in an explosion of energy datasets. Processing such data is challenging and usually takes a longer time than the requirement of a short-term load forecast. The paper addresses this concern by utilizing parallel computing capabilities to minimize the execution time while maintaining highly accurate load forecasting models. In this paper, a thousand smart meter energy datasets are analyzed to perform day ahead, hourly short-term load forecast (STLF). The paper utilizes multi-processing to enhance the overall execution time of the forecasting models by submitting simultaneous jobs to all the processors available. The paper demonstrates the efficacy of the proposed approach through the choice of machine learning (ML) models, execution time, and scalability. The proposed approach is validated on real energy consumption data collected at distribution transformers' level in Spanish Electrical Grid. Decision trees have outperformed the other models accomplishing a tradeoff between model accuracy and execution time. The methodology takes only 4 minutes to train 1,000 transformers for an hourly day-ahead forecast of (~24 million records) utilizing 32 processors.

Data is available to us in humongous amounts in the real world, but none of it is of practical use if not converted to useful information. However, the knowledge discovery is hindered because the real data is often incomplete and noisy. Nowadays, the problem of recovering missing data has found most important place in the field of data mining. Filling the missing data is a significant task, as it is paramount to use all available data for the given datasets are generally very small. In this paper, we deal with the real data with many missing values. Furthermore, we deal with the given data in three phases. The first phase considers the concept of feature selection, while the second phase iteratively considers filling in the missing values using probabilistic approach, keeping in mind the fact that features can be either nominal or numerical. Finally, the third phase deals with correcting the missing values that have been filled in. In our work, we have compared two imputation methods for dealing with the missing data, namely k-NN imputation method and mean and median imputation method. As a result, we have found that both of the imputation methods are efficient and yield more or less the same accuracy.