To realise the intelligent decision-making of dynamic scheduling and reconfiguration, we studied the intelligent scheduling and reconfiguration with dynamic job arrival for a reconfigurable flow line (RFL) using deep reinforcement learning (DRL), for the first time. The system architecture of intelligent scheduling and reconfiguration in smart manufacturing is proposed, and the mathematical model is established to minimise total tardiness cost. In addition, a DRL system of scheduling and reconfiguration is proposed by designing state features, actions, and rewards for scheduling and reconfiguration agents. Moreover, the advantage actor-critic (A2C) is adapted to solve the studied problem. The training curve shows the A2C-based agents have effectively learned to generate better solutions for unseen instances. The test results show that the A2C-based approach outperforms two traditional meta-heuristics, iterated greedy (IG) and genetic algorithm (GA), in solution quality and CPU times by a large margin. Specifically, the A2C-based approach outperforms IG and GA by 57.43% and 88.30%, using only 0.46‱ and 2.20‱ CPU times of IG and GA. The trained model can generate a scheduling or reconfiguration decision within 1.47 ms, which is almost instantaneous and can satisfy real-time optimisation. Our work shows a promising prospect of using DRL for intelligent scheduling and reconfiguration.

The scheduling problems of distributed production have received increasing attention in recent years. To coordinate the scheduling of production and transportation, we propose a novel distributed assembly permutation flowshop scheduling problem with flexible assembly and batch delivery (DAPFSP-FABD). The objective is to minimise the total cost of delivery and tardiness. Given that the problem involves six successive decisions, several batch allocation strategies for both production and distribution are proposed, and four neighbourhood structures are designed to search for the best batch sequence, batch allocation, product sequence, and job sequence. Seven algorithms, including four heuristics, a variable neighbourhood descent (VND) algorithm, and two iterated greedy (IG) algorithms, are proposed. In the IG, two types of destruction and construction operations are present to facilitate escape from local optima, thereby resulting in two IG algorithms: IG_desP and IG_desJ. Finally, extensive computational and statistical experiments are carried out. The results show that the proposed batch allocation strategies can improve the solution significantly. Additionally, the heuristics can obtain reasonable solutions in a short time, while IG_desJ performs the best regarding the quality of solutions. This study can help cross-regional companies coordinate the scheduling of production and transportation and enhance the performance of supply chains.

Dynamic scheduling problems have been receiving increasing attention in recent years due to their practical implications. To realize real-time and the intelligent decision-making of dynamic scheduling, we studied dynamic permutation flowshop scheduling problem (PFSP) with new job arrival using deep reinforcement learning (DRL). A system architecture for solving dynamic PFSP using DRL is proposed, and the mathematical model to minimize total tardiness cost is established. Additionally, the intelligent scheduling system based on DRL is modeled, with state features, actions, and reward designed. Moreover, the advantage actor-critic (A2C) algorithm is adapted to train the scheduling agent. The learning curve indicates that the scheduling agent learned to generate better solutions efficiently during training. Extensive experiments are carried out to compare the A2C-based scheduling agent with every single action, other DRL algorithms, and meta-heuristics. The results show the well performance of the A2C-based scheduling agent considering solution quality, CPU times, and generalization. Notably, the trained agent generates a scheduling action only in 2.16 ms on average, which is almost instantaneous and can be used for real-time scheduling. Our work can help to build a self-learning, real-time optimizing, and intelligent decision-making scheduling system.