For effective engagement of residential demand-side resources and to ensure efficient operation of distribution networks, we must overcome the challenges of controlling and coordinating residential components and devices at scale. In this paper, we present a distributed and scalable algorithm with a three-level hierarchical information exchange architecture for managing the residential demand response programs. First, a centralized optimization model is formulated to maximize community social welfare. Then, this centralized model is solved in a distributed manner with alternating direction method of multipliers (ADMM) by decomposing the original problem to utility-level and house-level problems. The information exchange between the different layers is limited to the primary residual (i.e., supply-demand mismatch), Lagrangian multipliers, and the total load of each house to protect each customer’s privacy. Simulation studies are performed on the IEEE 33 bus test system with 605 residential customers. The results demonstrate that the proposed approach can reduce customers’ electricity bills and reduce the peak load at the utility level without much affecting customers’ comfort and privacy. Finally, a quantitative comparison of the distributed and centralized algorithms shows the scalability advantage of the proposed ADMM-based approach, and it gives benchmarking results with achievable value for future research works.

Intelligent Heating, Ventilation, and Air Conditioning (HVAC) control using deep reinforcement learning (DRL) has recently gained a lot of attention due to its ability to optimally control the complex behavior of the HVAC system. However, more exploration is needed on understanding the adaptability challenges that the DRL agent could face during the deployment phase. Using online learning for such applications is not realistic due to the long learning period and likely poor comfort control during the learning process. Alternatively, DRL can be pre-trained using a building model prior to deployment. However, developing an accurate building model for every house and deploying a pre-trained DRL model for HVAC control would not be cost-effective. In this study, we focus on evaluating the ability of DRL-based HVAC control to provide cost savings when pre-trained on one building model and deployed on different house models with varying user comforts. We observed around 30% of cost reduction by pre-trained model over baseline when validated in a simulation environment and achieved up to 21% cost reduction when deployed in the real house. This finding provides experimental evidence that the pre-trained DRL has the potential to adapt to different house environments and comfort settings.

The implementations of residential demand response (DR) based on heating, ventilation, and air conditioning (HVAC) are inseparable from effective control algorithms for coordinating the operating schedules of multiple HVAC devices. In this work, both model-based and data-driven HVAC control strategies are developed to determine the optimal control actions for HVAC systems. The control objectives are to minimize customers' electricity costs, customers' discomfort, and the utility-level load violation. In the model-based approach, a thermal resistance-capacitance (RC) HVAC model is formulated to capture buildings' thermodynamic behaviors, and a distributed solution algorithm (i.e., alternating direction method of multipliers) is applied to determine the day-ahead HVAC operation schedules. In the data-driven approach, the neural networks continuously interact with the environment during the training process to learn what control actions to take under certain circumstances and then are used for online decision-making. The case study is performed on a utility system with one hundred houses. Simulation results demonstrate that the model-based approach can save 22% of the total cost compared to the data-driven approach, while the data-driven approach does not require outdoor temperature forecast information and its computational speed is 46 times faster than that of the model-based approach.

This paper presents a comprehensive scheduling framework for residential demand response (DR) programs considering both the day-ahead and real-time electricity markets. In the first stage, residential customers determine the operating status of their responsive devices such as heating, ventilation, and air conditioning (HVAC) systems and electric water heaters (EWHs), while the distribution system operator (DSO) computes the amount of electricity to be purchased in the day-ahead electricity market. In the second stage, the DSO purchases insufficient (or sells surplus) electricity in the real-time electricity market to maintain the supply-demand balance. Due to its computational complexity and data privacy issues, the proposed model cannot be directly solved in a centralized manner, especially with a large number of uncertain scenarios. Therefore, this paper proposes a combination of stochastic programming (SP) and the alternating direction method of multipliers (ADMM) algorithm, called SP-ADMM, to decompose the original model and then solve each sub-problem in a distributed manner while considering multiple uncertain scenarios. The simulation study is performed on the IEEE 33-bus system including 121 residential houses. The results demonstrate the effectiveness of the proposed approach for large-scale residential DR applications under weather and consumer uncertainties.

Electric water heaters represent 14% of the electricity consumption in residential buildings. An average household in the United States (U.S.) spends about USD 400–600 (0.45 ¢/L–0.68 ¢/L) on water heating every year. In this context, water heaters are often considered as a valuable asset for Demand Response (DR) and building energy management system (BEMS) applications. To this end, this study proposes a model-free deep reinforcement learning (RL) approach that aims to minimize the electricity cost of a water heater under a time-of-use (TOU) electricity pricing policy by only using standard DR commands. In this approach, a set of RL agents, with different look ahead periods, were trained using the deep Q-networks (DQN) algorithm and their performance was tested on an unseen pair of price and hot water usage profiles. The testing results showed that the RL agents can help save electricity cost in the range of 19% to 35% compared to the baseline operation without causing any discomfort to end users. Additionally, the RL agents outperformed rule-based and model predictive control (MPC)-based controllers and achieved comparable performance to optimization-based control.

The world of IoT (internet of things) is spawning new control options and decision making that has not been available before. This is providing a wealth of opportunities for transactive type controls and systems that can negotiate for a common goal. This paper discusses a transactive residential neighborhood with optimization at the home level utilizing a system of agents. The real-time optimization utilizes information and modeling to optimize heat pump water heater operation in occupied homes. Data is presented showing the performance of the optimization and conclusions are drawn on next steps for development.