23 December 2020 in IEEE Access
Rapid advances of sensing and cloud technologies transform the manufacturing system into a data-rich environment and make production scheduling increasingly complex. Traditional offline scheduling methods are limited in the ability to handle low-volume-high-mix workorders with diverse design specifications. Simulation-based methods show the promise for distributed scheduling of manufacturing jobs but are mostly implemented with historical data and empirical rules in a static manner. Recently, artificial intelligence (AI) algorithms fuel increasing interests to solve dynamic scheduling problems in the manufacturing setting. However, it’s difficult to utilize high-dimensional data for production scheduling while considering multiple practical objectives for smart manufacturing (e.g., minimize the makespan, reduce production costs, balance workloads). Therefore, this paper presents a new AI scheduler with composite reward functions for data-driven dynamic scheduling of manufacturing jobs under uncertainty in a smart factory. Internet-enabled sensor networks are deployed in the smart factory to track real-time statuses of workorders, machines, and material handling systems. A novel manufacturing value network is developed to take high-dimensional data as the input and then learn the state-action values for real-time decision making. Based on reinforcement learning (RL), composite rewards help the AI scheduler learn efficiently to achieve multiple objectives for production scheduling in real time. The proposed methodology is evaluated and validated with experimental studies in a smart manufacturing setting. Experimental results show that the new AI scheduler not only improves the multi-objective performance metrics in the production scheduling problem but also effectively copes with unexpected events (e.g., urgent workorders, machine failures) in manufacturing systems.
12 March 2022 in Advances in Mechanical Engineering
The job-shop scheduling problem (JSSP) is a complex combinatorial problem, especially in dynamic environments. Low-volume-high-mix orders contain various design specifications that bring a large number of uncertainties to manufacturing systems. Traditional scheduling methods are limited in handling diverse manufacturing resources in a dynamic environment. In recent years, artificial intelligence (AI) arouses the interests of researchers in solving dynamic scheduling problems. However, it is difficult to optimize the scheduling policies for online decision making while considering multiple objectives. Therefore, this paper proposes a smart scheduler to handle real-time jobs and unexpected events in smart manufacturing factories. New composite reward functions are formulated to improve the decision-making abilities and learning efficiency of the smart scheduler. Based on deep reinforcement learning (RL), the smart scheduler autonomously learns to schedule manufacturing resources in real time and improve its decision-making abilities dynamically. We evaluate and validate the proposed scheduling model with a series of experiments on a smart factory testbed. Experimental results show that the smart scheduler not only achieves good learning and scheduling performances by optimizing the composite reward functions, but also copes with unexpected events (e.g. urgent or simultaneous orders, machine failures) and balances between efficiency and profits.