This work proposes a novel deep reinforcement learning (DRL)-based demand response algorithm for smart facilities energy management to minimize electricity costs while maintaining a satisfaction index. Specifically, to accommodate the characteristics of the decision-making problem, long short-term memory (LSTM) units are adopted to extract discriminative features from past electricity price sequences and fed into fully connected multi-layer perceptrons (MLPs) with the measured energy and time information, then a deep Q-network is developed to approximate the optimal policy. After that, an experimental setup is constructed to investigate the effectiveness of the proposed DRL-based demand response algorithm to bridge the gap between theoretical studies and practical implementations. Numerical results demonstrate that the proposed algorithm can handle energy management well for multiple smart facilities. Moreover, the proposed algorithm outperforms the model predictive control (MPC) strategy and uncontrolled solution and is close to the theoretical optimal control method.

As a major consumer of energy, the industrial sector must assume the responsibility for improving energy efficiency and reducing carbon emissions. However, most existing studies on industrial energy management are suffering from modeling complex industrial processes. To address this issue, a model-free demand response (DR) scheme for industrial facilities was developed. In practical terms, we first formulated the Markov decision process (MDP) for industrial DR, which presents the composition of the state, action, and reward function in detail. Then, we designed an actor-critic-based deep reinforcement learning algorithm to determine the optimal energy management policy, where both the actor (Policy) and the critic (Value function) are implemented by the deep neural network. We then confirmed the validity of our scheme by applying it to a real-world industry. Our algorithm identified an optimal energy consumption schedule, reducing energy costs without compromising production.

In modern manufacturing systems, various industrial communication systems (e.g., fieldbus systems and industrial Ethernet networks) have been used to realize reliable information exchange. However, these industrial communication solutions are largely incompatible with each other, which do not satisfy the new requirements of Industry 4.0. Recently Time-Sensitive Networking (TSN) has been developed to improve the real-time capabilities to the standard Ethernet, and is considered to be a promising real-time communication solution for Industry 4.0. In this work, we propose a communication architecture for a manufacturing system using the Open Platform Communications Unified Architecture (OPC UA) and TSN technologies. TSN is adopted as the communication backbone to connect heterogeneous industrial automation subsystems. The OPC UA is adopted to realize horizontal and vertical communication between subsystems in the field layer and the entities of the upper layers. We implement a laboratory-level manufacturing system to validate the proposed architecture. The experimental results demonstrate the feasibility and capability of the proposed architecture. Moreover, we evaluate the performance of a key TSN substandard, i.e., IEEE 802.1Qbv, in the laboratory-level manufacturing system. The evaluation results demonstrate that IEEE 802.1Qbv can indeed provide excellent real-time capabilities for industrial applications.