Intelligent Heating, Ventilation, and Air Conditioning (HVAC) control using deep reinforcement learning (DRL) has recently gained a lot of attention due to its ability to optimally control the complex behavior of the HVAC system. However, more exploration is needed on understanding the adaptability challenges that the DRL agent could face during the deployment phase. Using online learning for such applications is not realistic due to the long learning period and likely poor comfort control during the learning process. Alternatively, DRL can be pre-trained using a building model prior to deployment. However, developing an accurate building model for every house and deploying a pre-trained DRL model for HVAC control would not be cost-effective. In this study, we focus on evaluating the ability of DRL-based HVAC control to provide cost savings when pre-trained on one building model and deployed on different house models with varying user comforts. We observed around 30% of cost reduction by pre-trained model over baseline when validated in a simulation environment and achieved up to 21% cost reduction when deployed in the real house. This finding provides experimental evidence that the pre-trained DRL has the potential to adapt to different house environments and comfort settings.

Phase change material (PCM)-based thermal energy storage (TES) can provide energy and cost savings and peak demand reduction benefits for grid-interactive residential buildings. Researchers established that these benefits vary greatly depending on the PCM phase change temperature (PCT), total TES storage capacity, system configuration and location and climate of the building. In this study, preliminary techno-economic performance is reported for a novel heat pump (HP)-integrated TES system using an idealized approach. A simplified HP-TES was modeled for 1 year of space heating and cooling loads for a residential building in three different climates in the United States. The vapor compression system of the HP was modified to integrate with TES, and all heat transfer to and from the TES was mediated by the HP. A single PCM was used for heating and cooling, and the PCT and TES capacity were varied to observe their effects on the building’s energy consumption, peak load shifting and cost savings. The maximum reduction in electric consumption, utility cost and peak electric demand were achieved at a PCT of 30 °C for New York City and 20 °C for Houston and Birmingham. Peak energy consumption in Houston, New York City, and Birmingham was reduced by 47%, 53%, and 70%, respectively, by shifting peak load using a time-of-use utility schedule. TES with 170 MJ storage capacity allowed for maximum demand shift from on-peak to off-peak hours, with diminishing returns once the TES capacity equaled the daily building thermal loads experienced during the most extreme ambient conditions.