Heating, Ventilation, and Air Conditioning (HVAC) are extremely energy-consuming, accounting for 40% of total building energy consumption. It is crucial to design some energy-efficient building thermal comfort control strategy which can reduce the energy consumption of the HVAC while maintaining the comfort of the occupants. However, implementing such a strategy is challenging, because the changes of the thermal states in a building environment are influenced by various factors. The relationships among these influencing factors are hard to model and are always different in different building environments. To address this challenge, we propose a deep reinforcement learning based framework, DeepComfort, for thermal comfort control in buildings. We formulate the thermal comfort control as a cost-minimization problem by jointly considering the energy consumption of the HVAC and the occupants’ thermal comfort. We first design a deep Feedforward Neural Network (FNN) based approach for predicting the occupants’ thermal comfort, and then propose a Deep Deterministic Policy Gradients (DDPG) based approach for learning the optimal thermal comfort control policy. We implement a building thermal comfort control simulation environment and evaluate the performance under various settings. The experimental results show that our approaches can improve the performance of thermal comfort prediction by 14.5% and reduce the energy consumption of HVAC by 4.31% while improving the occupants’ thermal comfort by 13.6%.

Recently, many models have been developed to predict video Quality of Experience (QoE), yet the applicability of these models still faces significant challenges. Firstly, many models rely on features that are unique to a specific dataset and thus lack the capability to generalize. Due to the intricate interactions among these features, a unified representation that is independent of datasets with different modalities is needed. Secondly, existing models often lack the configurability to perform both classification and regression tasks. Thirdly, the sample size of the available dataset to develop these models is often very small, and the impact of limited datasets on the performance of QoE models has not been adequately addressed. To address these issues, in this work we develop a novel and end-to-end framework termed as DeepQoE. The proposed framework first uses a combination of deep learning techniques, such as word embedding and 3D convolutional neural network (C3D), to extract generalized features. Next, these features are combined and fed into a neural network for representation learning. A learned representation will then serve as input for classification or regression tasks. We evaluate the performance of DeepQoE with three datasets. The results show that for small datasets (e.g., WHU-MVQoE2016 and Live-Netflix Video Database), the performance of state-of-the-art machine learning algorithms is greatly improved by using the QoE representation from DeepQoE (e.g., 35.71% to 44.82%); while for the large dataset (e.g., VideoSet), our DeepQoE framework achieves significant performance improvement in comparison to the best baseline method (90.94% vs. 82.84%). In addition to the much improved performance, DeepQoE has the flexibility to fit different datasets, to learn QoE representation, and to perform both classification and regression problems. We also develop a DeepQoE based adaptive bitrate streaming (ABR) system to verify that our framework can be easily applied to multimedia communication service. The software package of the DeepQoE framework has been released to facilitate the current research on QoE.

Renewable energy technologies empower microgrids to generate electricity to supply themselves and trade with others. Under this paradigm, microgrids have become autonomous entities that must intelligently determine their policies for energy trading and scheduling. Many factors influence a microgrid's decision-making, such as the complex microgrid infrastructure, the uncertain energy yield and demand, and the competition among the energy market players. These factors are usually hard to precisely model, and deriving the optimal policy for a microgrid is challenging. We propose a multiagent reinforcement learning (MARL) approach with an attention mechanism to learn the optimal policies for the microgrids without complex system modeling. We model each microgrid as an autonomous agent, which learns how to schedule energy resources and trade with others by collaborating with other agents. We adopt attention mechanism to enable intelligently selecting contextual information for the training of each agent. After training, an agent can make control decisions using only its local information, which can well preserve the microgrids' privacy and reduce the communication overhead among microgrids to facilitate distributed control. We implement a simulation environment and evaluate the performances of our proposed method using real-world datasets. The experimental results show that our method can significantly reduce the cost of the microgrids compared with the baseline methods.

Microgrids, gaining traction from rising distributed generation for carbon reduction, demand novel solutions to regulate on- and off-grid operations, as well as both energy and monetary transfers between the microgrid and the central grid and among different microgrid participants. This research aims to develop and validate an intelligent microgrid management system to secure the competitiveness of Singapore’s energy market, by leveraging the inherent synergy between two emerging technologies, i.e., blockchain for Peer-to-Peer (P2P) solar power trading and fog computing for grid infrastructure management. For this vision, we have developed FogChain, an integrative, cost-effective, and scalable microgrid operating system (MGOS), consisting of three technical service layers: 1) a novel microgrid information infrastructure based on the fog-computing paradigm (i.e., intelligence on edge); 2) a blockchain-based microgrid service layer, providing smart contract and decentralized control capabilities for grid application development; and 3) a microgrid application layer (i.e., P2P energy trading) over the blockchain-based grid service. This MGOS would fundamentally transform how solar power is traded among participating electricity prosumers, leading to potentially new operational and business models. We have implemented the FogChain system and conducted extensive experiments to verify its performance advantages. Our results demonstrate that FogChain can efficiently process energy auction among 1000 participants with 1.1 s delay on average, reduce transmission cost up to 20% under the loss-aware trading mechanism, and reduce the solar yield prediction error to 0.11. Our system prototype suggests that FogChain provides a promising solution for efficient decentralized energy trading and intelligent distributed control for microgrids.

Data drift is a thorny challenge when deploying person re-identification (ReID) models into real-world devices, where the data distribution is significantly different from that of the training environment and keeps changing. To tackle this issue, we propose a federated spatial-temporal incremental learning approach, named FedSTIL, which leverages both lifelong learning and federated learning to continuously optimize models deployed on many distributed edge clients. Unlike previous efforts, FedSTIL aims to mine spatial-temporal correlations among the knowledge learnt from different edge clients. Specifically, the edge clients first periodically extract general representations of drifted data to optimize their local models. Then, the learnt knowledge from edge clients will be aggregated by centralized parameter server, where the knowledge will be selectively and attentively distilled from spatial- and temporal-dimension with carefully designed mechanisms. Finally, the distilled informative spatial-temporal knowledge will be sent back to correlated edge clients to further improve the recognition accuracy of each edge client with a lifelong learning method. Extensive experiments on a mixture of five real-world datasets demonstrate that our method outperforms others by nearly 4% in Rank-1 accuracy, while reducing communication cost by 62%. All implementation codes are publicly available on https://github.com/MSNLAB/Federated-Lifelong-Person-ReID.

Energy usage and thermal comfort are the pillars of smart buildings. Many research works have been proposed to save energy while maintaining a comfortable thermal condition. However, most of them either make the oversimplified assumption on thermal comfort with unsatisfied comfort performance or deal with the single-zone thermal control only with limited practical impact. A few preliminary pieces of research on multizone control are available, but they fail to keep pace with the latest advancements in the deep-learning-based control techniques. In this article, we investigate the multizone thermal control with optimized energy usage and canonical thermal comfort modeling. We adopt the emerging multiagent deep reinforcement learning techniques and propose to model each zone as an agent. A multiagent framework is established to support the information exchange among the agents and enable intelligent thermal control in the heterogeneous zones. Accordingly, we mathematically formulate a problem to optimize both energy and comfort. A multizone thermal control algorithm (MOCA) is proposed to solve the problem by deriving optimal control policies. We validate the performance of MOCA through simulation in professional TRNSYS, configured based on our real-world laboratory. The results are promising with up to 15.4% energy saving as well as satisfied thermal comfort in different zones.