In real-life production systems, arrivals of jobs are usually unpredictable, which makes it necessary to develop solid reactive scheduling policies to meet delivery requirements. Deep reinforcement learning (DRL) based scheduling methods are capable of quickly responding to dynamic events by learning from the training data. However, most of policy networks in DRL algorithms are trained to choose priority dispatching rules (PDR), thus, to some extent, the efficiency of obtained scheduling plans is limited by the performance of PDRs. This paper investigates a dynamic flexible job shop scheduling problem with random job arrivals for the total tardiness minimization. A DRL-based reactive scheduling method, proximal policy optimization with attention-based policy network (PPO-APN), is proposed to make real-time decisions for the dynamic scheduling environment, where the attention-based policy network (APN) is able to directly select pending jobs distinguished from the action space that consists of PDRs. Additionally, a global/local reward function (GLRF) is designed to address the reward sparsity issue during training processes. The proposed PPO-APN is tested on randomly generated instances with different production configurations, and is compared with frequently-used PDRs and DRL-based methods. Numerical experimental results indicate that APN and GLRF components significantly improve the training efficiency, and the PPO-APN shows better overall performance compared with other methods. Note to Practitioners —This work is motivated by a typical production scenario in discrete manufacturing systems, where orders randomly arrive at the shop floor and require to be scheduled in a short time to ensure the on-time delivery. Previous research work tends to apply DRL algorithms to choose suitable dispatching rules for the ease of implementation. Nevertheless, the jobs that can be selected by dispatching rules are rather limited, thus many possible high-quality scheduling plans are ignored. This work first sorts all the unscheduled jobs by a heuristic algorithm, and puts some of top-ranked jobs to a pool. When a machine becomes available, it will directly choose a job from the pool as the next processing task. The job selection policy is represented by a novel attention-based network, and is trained by a powerful DRL algorithm. The aforementioned process is repeatedly executed in a simulation environment to collect the training data. Therefore, after being trained for a certain period of time, the policy will become smarter and can be applied to make right decisions in real-time. The proposed reactive scheduling method has been proved to be more efficient than dispatching rules and DRL-based approaches, and is effective in the production scheduling for a wide variety of discrete manufacturing scenarios, such as automobile and electronics industries. Moreover, the proposed method can be further extended to address dynamic scheduling problems with some production characteristics via adding constraints for the job selection or re-defining calculations for the completion time of operations accordingly.

As an extension of the classical flow-shop scheduling problem, the hybrid flow-shop scheduling problem (HFSP) widely exists in large-scale industrial production systems and has been considered to be challenging for its complexity and flexibility. Evolutionary algorithms based on encoding and heuristic decoding approaches are shown effective in solving the HFSP. However, frequently used encoding and decoding strategies can only search a limited area of the solution space, thus leading to unsatisfactory performance during the later period. In this article, a hybrid evolutionary algorithm (HEA) using two solution representations is proposed to solve the HFSP for makespan minimization. First, the proposed HEA searches the solution space by a permutation-based encoding representation and two heuristic decoding methods to find some promising areas. Afterward, a Tabu search (TS) procedure based on a disjunctive graph representation is introduced to expand the searching space for further optimization. Two classical neighborhood structures focusing on critical paths are extended to the problem-specific backward schedules to generate candidate solutions for the TS. The proposed HEA is tested on three public HFSP benchmark sets from the existing literature, including 567 instances in total, and is compared with some state-of-the-art algorithms. Extensive experimental results indicate that the proposed HEA performs much better than the other algorithms. Moreover, the proposed method finds new best solutions for 285 hard instances.

Multi-variety and small-batch production mode enables manufacturing industries to expeditiously satisfy customers' personalised demands, where a large amount of identical jobs can be split into several sublots, and be processed by reconfigurable machines with multiple machining technics. However, such highly flexible manufacturing environments bring some intractable problems to the production scheduling. Mathematical programming and meta-heuristic methods become less efficient when a scheduling problem contains both discrete and continuous optimisation attributes. Therefore, matheuristic, which combines advantages of the two methodologies, is regarded as a promising solution. This paper investigates a flexible job shop scheduling problem with lot-streaming and machine reconfigurations (FJSP-LSMR) for the total weighted tardiness minimisation. First, a monolithic mixed integer linear programming (MILP) model is established for the FJSP-LSMR. Afterwards, a matheuristic method with a variable neighbourhood search component (MH-VNS) is developed to address the problem. The MH-VNS adopts the classical genetic algorithm (GA) as the framework, and introduces two MILP-based lot-streaming optimisation strategies, LSO1 and LSO2, to improve lot-sizing plans with varying degrees. Four groups of instances are extended from the well-known Fdata benchmark to evaluate the performance of proposed MILP model, LSO1 and LSO2 components, and MH-VNS. Numerical experimental results suggest that LSO1 and LSO2 are efficient in different scenarios, and the proposed MH-VNS can well balance the solution quality and computational costs for reasonably integrating the GA- and MILP-based local search strategies. In addition, a complicated FJSP-LSMR case is abstracted from a real-world shop floor for processing large-sized structural parts to further validate the MH-VNS.

Evolutionary algorithms and swarm intelligence algorithms have been widely used for constrained optimization problems for decades and numerous techniques for constraint handling have been proposed. The ${\varepsilon }$ -constrained method is a very effective one. In the literature, the ${\varepsilon }$ value was usually controlled via an exponential function, which is not competent for solving certain types of constrained optimization problems, e.g., whose global optima are located near the boundary of the feasible and infeasible regions. To solve this problem, this article proposes a new adaptive ${\varepsilon }$ control method and incorporate it into a basic differential evolution (DE) algorithm: (DE/rand/1/exp). Based on the information of constraint violation in the current population, the adaptive method controls the value of ${\varepsilon }$ through a simple heuristic rule. Compared with the traditional exponential function-based control methods, the proposed adaptive method can prevent the algorithm from being trapped into local optima while retaining the obtained near-optimal candidate solutions in the infeasible region for generating promising searching paths. Besides, we set the crossover rate (CR) as a more reasonable value for DE/rand/1/exp, which can enhance the efficiency significantly. The well-known 2006 IEEE Congress on Evolutionary Computation (CEC 2006) competition on real-parameter single-objective constrained optimization benchmark is adopted to evaluate the effectiveness of the proposed adaptive ${\varepsilon }$ -constrained DE. Fifteen constrained engineering optimization problems are collected from the literature to test the proposed algorithm. Moreover, the adaptive ${\varepsilon }$ control method is extended to an adaptive algorithm to solve the benchmark problems from CEC 2017. The comparison results confirm the superiority of the proposed method.

When a multiobjective evolutionary algorithm based on decomposition (MOEA/D) is applied to solve problems with discontinuous Pareto front (PF), a set of evenly distributed weight vectors may lead to many solutions assembling in boundaries of the discontinuous PF. To overcome this limitation, this article proposes a mechanism of resetting weight vectors (RWVs) for MOEA/D. When the RWV mechanism is triggered, a classic data clustering algorithm DBSCAN is used to categorize current solutions into several parts. A classic statistical method called principal component analysis (PCA) is used to determine the ideal number of solutions in each part of PF. Thereafter, PCA is used again for each part of PF separately and virtual targeted solutions are generated by linear interpolation methods. Then, the new weight vectors are reset according to the interrelationship between the optimal solutions and the weight vectors under the Tchebycheff decomposition framework. Finally, taking advantage of the current obtained solutions, the new solutions in the decision space are updated via a linear interpolation method. Numerical experiments show that the proposed MOEA/D-RWV can achieve good results for bi-objective and tri-objective optimization problems with discontinuous PF. In addition, the test on a recently proposed MaF benchmark suite demonstrates that MOEA/D-RWV also works for some problems with other complicated characteristics.