With the development of the Internet of Things (IoT) technologies, implementing intelligent controls in buildings to reduce energy consumption is becoming increasingly popular. Building climate control is of strong research interest due to high energy savings potentials associated with heating, ventilation and air-conditioning (HVAC) controls. Because the operation of HVAC systems directly influences occupant thermal comfort, building-specific thermal models are needed for proper control. Such models describe how indoor temperature changes under different environmental conditions and HVAC status. Previous studies mainly developed thermal models for accurate building thermal simulation which are not practical for use in real-world applications as they require domain knowledge and lengthy building-by-building configuration. In this paper, taking advantage of IoT technologies, a plug & play learning framework is proposed to automatically identify the thermal model of each thermal zone in a building without manual configuration. In contrast to the existing methods, a thermal model is learnt using low-resolution temperature readings from IoT-based smart thermostats. The learning framework has been validated using data collected from an IoT platform installed in a building over a summer. The performance of the learnt model and the error of indoor temperature prediction based on this model have been evaluated and quantified. Findings demonstrate the learning process can be automated with either edge-computing or cloud-computing. This study shows the validity of the proposed IoT-based thermal model learning framework, and offers a pragmatic solution for providing reliable thermal models to future smart building climate controls.

Buildings, as major energy consumers, can provide great untapped demand response (DR) resources for grid services. However, their participation remains low in real-life. One major impediment for popularizing DR in buildings is the lack of cost-effective automation systems that can be widely adopted. Existing optimization-based smart building control algorithms suffer from high costs on both building-specific modeling and on-demand computing resources. To tackle these issues, this paper proposes a cost-effective edge-cloud integrated solution using reinforcement learning (RL). Beside RL’s ability to solve sequential optimal decision-making problems, its adaptability to easy-to-obtain building models and the off-line learning feature are likely to reduce the controller’s implementation cost. Using a surrogate building model learned automatically from building operation data, an RL agent learns an optimal control policy on cloud infrastructure, and the policy is then distributed to edge devices for execution. Simulation results demonstrate the control efficacy and the learning efficiency in buildings of different sizes. A preliminary cost analysis on a 4-zone commercial building shows the annual cost for optimal policy training is only 2.25% of the DR incentive received. Results of this study show a possible approach with higher return on investment for buildings to participate in DR programs.

Maintaining an accurate power system model is crucial for a variety of applications such as electricity markets. In this paper, the cyber security issue induced by critical parameters in electricity market operations is investigated. Critical parameters are a set of model parameters the errors of which cannot be detected due to the lack of local measurements. In this paper, it is first proven that the criticality property of parameters is independent from system operating points, i.e. the maliciously injected errors will always remain undetectable. Then, the cyber attack against critical parameters is formulated as a bilevel programming problem in search of the optimal attack strategy or the worst situation faced by the system operators. An effective algorithm for solving this problem is also provided. Simulation results verify that by injecting undetectable errors into critical model parameters, the locational marginal prices in electricity markets can be significantly distorted, which leads to financial benefits for the attackers.

We propose a novel data-driven method to accelerate the convergence of Alternating Direction Method of Multipliers (ADMM) for solving distributed DC optimal power flow (DC-OPF) where lines are shared between independent network partitions. Using previous observations of ADMM trajectories for a given system under varying load, the method trains a recurrent neural network (RNN) to predict the converged values of dual and consensus variables. Given a new realization of system load, a small number of initial ADMM iterations is taken as input to infer the converged values and directly inject them into the iteration. We empirically demonstrate that the online injection of these values into the ADMM iteration accelerates convergence by a significant factor for partitioned 14-, 118- and 2848-bus test systems under differing load scenarios. The proposed method has several advantages: it maintains the security of private decision variables inherent in consensus ADMM; inference is fast and so may be used in online settings; RNN-generated predictions can dramatically improve time to convergence but, by construction, can never result in infeasible ADMM subproblems; it can be easily integrated into existing software implementations. While we focus on the ADMM formulation of distributed DC-OPF in this letter, the ideas presented are naturally extended to other distributed optimization problems.

This paper focuses on the critical load restoration problem in distribution systems following major outages. To provide fast online response and optimal sequential decision-making support, a reinforcement learning (RL) based approach is proposed to optimize the restoration. Due to the complexities stemming from the large policy search space, renewable uncertainty, and nonlinearity in a complex grid control problem, directly applying RL algorithms to train a satisfactory policy requires extensive tuning to be successful. To address this challenge, this paper leverages the curriculum learning (CL) technique to design a training curriculum involving a simpler steppingstone problem that guides the RL agent to learn to solve the original hard problem in a progressive and more effective manner. We demonstrate that compared with direct learning, CL facilitates controller training to achieve better performance. To study realistic scenarios where renewable forecasts used for decision-making are in general imperfect, the experiments compare the trained RL controllers against two model predictive controllers (MPCs) using renewable forecasts with different error levels and observe how these controllers can hedge against the uncertainty. Results show that RL controllers are less susceptible to forecast errors than the baseline MPCs and can provide a more reliable restoration process.

This paper develops an intelligent grid-interactive building controller, which optimizes building operation during both normal hours and demand response (DR) events. To avoid costly on-demand computation and to adapt to non-linear building models, the controller utilizes reinforcement learning (RL) and makes real-time decisions based on a near-optimal control policy. Learning such a policy typically amounts to solving a hard non-convex optimization problem. We propose to address this problem with a novel global-local policy search method. In the first stage, an RL algorithm based on zero-order gradient estimation is leveraged to search for the optimal policy globally, due to its scalability and the potential to escape some poor performing local optima. The obtained policy is then fine-tuned locally to bring the first-stage solution closer to that of the original unsmoothed problem. Experiments on a simulated five-zone commercial building demonstrate the advantages of the proposed method over existing learning approaches. They also show that the learned control policy outperforms a pragmatic linear model predictive controller (MPC) and approaches the performance of an oracle MPC in testing scenarios. Using a state-of-the-art advanced computing system, we demonstrate that the controller can be learned and deployed within hours of training.