A data-based Stackelberg market strategy for a distribution market operator (DMO) is proposed to coordinate power dispatch among different virtual power plants (VPP), i.e., demand response aggregators (DRA). The proposed strategy has a two-stage framework. In the first stage, a data-driven method based on noisy inverse optimization estimates the complicated price-response characteristics of customer loads. The estimated load information of the DRAs is delivered to the second stage where a one-leader multiple-follower stochastic Stackelberg game is formulated to represent the practical market interaction between the DMO and the DRAs that considers the uncertainty of renewables and the operational security. The proposed data-driven game model is solved by a new penalty algorithm and a customized distributed hybrid dual decomposition-gradient descent (HDDGD) algorithm. Case studies on a practical demand response project in China and a distribution test system demonstrate the effectiveness of the proposed methodology.

Power quality (PQ) disturbance in power systems has been a concern for operators and customers. The purpose is to locate and forecast the presence of PQ disturbances to suppress or avoid their negative effects on power grid and appliances. This study, using a multi-hidden Markov model (MHMM), motivates data-driven tools to achieve situation awareness of PQ disturbance. We first design a modified adaptive-sorted neighborhood method that consists of blocking and merging phases to locate PQ disturbance sources from a large volume of PQ records. We then group and discretize the data on disturbance sources and weather conditions according to regions. The capability of MHMM-based tools to predict future PQ disturbance level can be improved by clustering the training set of time series of PQ disturbance levels based on weather conditions. A Hadoop-based PQ analysis framework is proposed to reduce computational times, considering the volume of PQ data in a realistic power grid is large. We utilize numerical case studies that use real data collected from a power system of a Chinese city to investigate the correctness and feasibility of the proposed method.

Rapid growth of diversity, uncertainty, and coupling effect of units in modern energy systems jointly challenges the traditional model-based situation awareness (SA) in Energy Internet of Things (EIoT). This work explores the digital twin of EIoT (EIoT-DT) and then provides a novel data-driven SA paradigm, named DT-SA, as a promising alternative. Based on the combination of the latest data technologies and machine learning algorithms, DT-SA transfers those stubborn SA challenges to digital space, and then addresses them by building a domain-specific and data-friendly digital twin (DT) model upon massive data. The established model can be quantitatively tested via iterative virtual–real interaction and, thus, be evaluated and updated through closed-loop feedback to improve its performance in the physical world. To this end, some engineering and scientific problems are raised: 1) virtual–real interaction mechanism relevant to resource flow and data flow; 2) unified modeling and analysis of heterogeneous spatial–temporal data; 3) DT configuration and evolution; and 4) domain-specific DT-SA characterization. To solve these problems, cloud-edge-terminal configuration, big data analytics (BDA), DT, and SA indicator systems are studied, respectively. Then, the random matrix theory (RMT) and overarching DT-SA framework are designed as a roadmap. Besides, some potential applications and undergoing projects on the terminal, edge, or cloud are discussed, e.g., condition assessment of equipment, digital monitoring and diagnosis of the power grid network, and EIoT construction in the smart city. Finally, some perspectives and recommendations are proposed in conclusion for future research. This research can be regarded as an efficient handbook for both energy engineering and data science, which may benefit enterprise digitization, smart city, etc.

With the development of deregulated retail power markets, it is possible for end users equipped with smart meters and controllers to optimize their consumption cost portfolios by choosing various pricing plans from different retail electricity companies. This paper proposes a reinforcement learning-based decision system for assisting the selection of electricity pricing plans, which can minimize the electricity payment and consumption dissatisfaction for individual smart grid end user. The decision problem is modeled as a transition probability-free Markov decision process (MDP) with improved state framework. The proposed problem is solved using a Kernel approximator-integrated batch Q-learning algorithm, where some modifications of sampling and data representation are made to improve the computational and prediction performance. The proposed algorithm can extract the hidden features behind the time-varying pricing plans from a continuous high-dimensional state space. Case studies are based on data from real-world historical pricing plans and the optimal decision policy is learned without a priori information about the market environment. Results of several experiments demonstrate that the proposed decision model can construct a precise predictive policy for individual user, effectively reducing their cost and energy consumption dissatisfaction.

Cyber, especially false data injection attacks (FDIAs), are gradually becoming a kind of severe threat to modern power grids. This paper proposes a blind FDIA approach against the state estimation of power grids based on matrix reconstruction and subspace estimation. As a blind FDIA method, it requires no information about the system parameters and topology, but only needs a limited period of measurement data. In particular, this paper investigates the interference of random measurement noise for subspace based blind FDIA methods, and designs a scheme to alleviate the influence of this random noise in the process of FDIA. Compared with previous blind FDIA methods, the proposed method overcomes the shortcoming that measurement noise is difficult to address when data is not abundant. Thus the presented approach performs high successful rate of FDIA, and it is able to operate when measurement data is very limited. Besides, it performs great robustness to large-scale power grids and high-level measurement noise. Numerous cases demonstrate the effectiveness and advantages of the proposed method.

Invisible units mainly refer to small-scale units that are not monitored by, and thus are not visible to utilities. Integration of these invisible units into power systems does significantly affect the way in which a distribution grid is planned and operated. This paper, based on random matrix theory (RMT), proposes a statistical, data-driven framework to handle the massive grid data, in contrast to its deterministic, model-based counterpart. Combining the RMT-based data-mining framework with conventional techniques, some heuristics are derived as the solution to the invisible units detection and estimation task: linear eigenvalue statistic indicators (LESs) are suggested as the main ingredients of the solution; according to the statistical properties of LESs, the hypothesis testing is formulated to conduct change point detection in the high-dimensional space. The proposed method is promising for anomaly detection and pertinent to current distribution networks-it is capable of detecting invisible power usage and fraudulent behavior while even being able to locate the suspect's location. Case studies, using both simulated data and actual data, validate the proposed method.