Residential energy consumption continues to climb steadily, requiring intelligent energy management strategies to reduce power system pressures and residential electricity bills. However, it is challenging to design such strategies due to the random nature of electricity pricing, appliance demand and user behavior. This paper presents a novel reward shaping (RS)-based actor-critic deep reinforcement learning (ACDRL) algorithm to manage the residential energy consumption profile with limited information about the uncertain factors. Specifically, the interaction between the energy management center and various residential loads is modeled as a Markov decision process that provides a fundamental mathematical framework to represent the decision making in situations where outcomes are partially random and partially influenced by the decision maker control signals, in which the key elements containing the agent, environment, state, action, and reward are carefully designed, and the electricity price is considered as a stochastic variable. An RS-ACDRL algorithm is then developed, incorporating both the actor and critic network and an RS mechanism, to learn the optimal energy consumption schedules. Several case studies involving real-world data are conducted to evaluate the performance of the proposed algorithm. Numerical results demonstrate that the proposed algorithm outperforms state-of-the-art RL methods in terms of learning speed, solution optimality and cost reduction.

As a distributed computing paradigm, edge computing has become a key technology for providing timely services to mobile devices by connecting Internet of Things (IoT), cloud centers, and other facilities. By offloading compute-intensive tasks from IoT devices to edge/cloud servers, the communication and computation pressure caused by the massive data in Industrial IoT can be effectively reduced. In the process of computation offloading in edge computing, it is critical to dynamically make optimal offloading decisions to minimize the delay and energy consumption spent on the devices. Although there are a large number of task offloading-decision models, how to measure and evaluate the quality of different models and configurations is crucial. In this article, we propose a novel simulation platform named ChainFL, which can build an edge computing environment among IoT devices while being compatible with federated learning and blockchain technologies to better support the embedding of security-focused offloading algorithms. ChainFL is lightweight and compatible, and it can quickly build complex network environments by connecting devices of different architectures. Moreover, due to its distributed nature, ChainFL can also be deployed as a federated learning platform across multiple devices to enable federated learning with high security due to its embedded blockchain. Finally, we validate the versatility and effectiveness of ChainFL by embedding a complex offloading-decision model in the platform, and deploying it in an Industrial IoT environment with security risks.

Load forecasting is critical to the task of energy management in power systems, for example, balancing supply and demand and minimizing energy transaction costs. There are many approaches used for load forecasting such as the support vector regression (SVR), the autoregressive integrated moving average (ARIMA), and neural networks, but most of these methods focus on single-step load forecasting, whereas multistep load forecasting can provide better insights for optimizing the energy resource allocation and assisting the decision-making process. In this work, a novel sequence-to-sequence (Seq2Seq)-based deep learning model based on a time series decomposition strategy for multistep load forecasting is proposed. The model consists of a series of basic blocks, each of which includes one encoder and two decoders; and all basic blocks are connected by residuals. In the inner of each basic block, the encoder is realized by temporal convolution network (TCN) for its benefit of parallel computing, and the decoder is implemented by long short-term memory (LSTM) neural network to predict and estimate time series. During the forecasting process, each basic block is forecasted individually. The final forecasted result is the aggregation of the predicted results in all basic blocks. Several cases within multiple real-world datasets are conducted to evaluate the performance of the proposed model. The results demonstrate that the proposed model achieves the best accuracy compared with several benchmark models.

Neighbourhood Area Networks (NANs) are critical infrastructure in smart grid to support communications. With the development of wireless communication technologies, there is a great potential for large-scale wireless-mesh NANs to be deployed to smart grid systems. While there exist theoretical models of complex networks, they are not directly applicable in the generation of network topology for wireless-mesh NANs due to their lack of some specifications unique in smart grid, leading to difficulties in theoretical studies of smart grid NANs. Among various topological structures, scale-free networks are considered to be a potential candidate. Therefore, this paper presents a systematic approach to generate scale-free topology of wireless-mesh NANs in smart grid. A theoretical analysis confirms the scale-free feature of the network topology generated from the approach. To reveal the characteristics of our approach in comparison with widely used models of general complex networks, simulations are conducted on various network parameters including network diameter, average path length, clustering coefficient and fractal dimension. The simulation results show that our approach for generating wireless-mesh NAN topology is fundamentally different from these compared models, and thus is a new type of complex-network model. Specifically, our model is suitable for wireless-mesh NANs in smart grid.