As semiconductor manufacturers, recently, have focused on producing multichip products (MCPs), scheduling semiconductor manufacturing operations become complicated due to the constraints related to reentrant production flows, sequence-dependent setups, and alternative machines. At the same time, the scheduling problems need to be solved frequently to effectively manage the variabilities in production requirements, available machines, and initial setup status. To minimize the makespan for an MCP scheduling problem, we propose a setup change scheduling method using reinforcement learning (RL) in which each agent determines setup decisions in a decentralized manner and learns a centralized policy by sharing a neural network among the agents to deal with the changes in the number of machines. Furthermore, novel definitions of state, action, and reward are proposed to address the variabilities in production requirements and initial setup status. Numerical experiments demonstrate that the proposed approach outperforms the rule-based, metaheuristic, and other RL methods in terms of the makespan while incurring shorter computation time than the metaheuristics considered.
3 December 2021 in IEEE Transactions on Cybernetics
Reinforcement learning (RL) has emerged as a promising approach for scheduling semiconductor operations. Yet, it is still challenging to solve large-scale scheduling problems based on an RL method since learning complexity grows fast as the size of shop floor increases. This challenge becomes more apparent when solving the scheduling problems with a diverse number of job types, which leads to the difficulties in exploration and function approximation in RL. This article presents a scheduling method for semiconductor packaging facilities using deep RL in which an agent allocates a job to one of machines in a centralized manner. Specifically, a novel state representation is introduced to effectively accommodate the variations in the number of available machines and the production requirements. Furthermore, we propose a continuous representation of an action to maintain the size of the action space even when the numbers of jobs, machines, and operation types are subject to change. Extensive experiments on large-scale datasets demonstrate that the proposed method mostly outperforms the metaheuristics and rule-based methods, as well as the other RL approaches considered in terms of makespan while requiring much less computation time than the metaheuristics.
Parallel machine scheduling with sequence-dependent family setups has attracted much attention from academia and industry due to its practical applications. In a real-world manufacturing system, however, solving the scheduling problem becomes challenging since it is required to address urgent and frequent changes in demand and due-dates of products. To minimize the total tardiness of the scheduling problem, we propose a deep reinforcement learning (RL) based scheduling framework in which trained neural networks (NNs) are able to solve unseen scheduling problems without re-training even when such changes occur. Specifically, we propose state and action representations whose dimensions are independent of production requirements and due-dates of jobs while accommodating family setups. At the same time, an NN architecture with parameter sharing was utilized to improve the training efficiency. Extensive experiments demonstrate that the proposed method outperforms the recent metaheuristics, rule-based, and other RL-based methods in terms of total tardiness. Moreover, the computation time for obtaining a schedule by our framework is shorter than those of the metaheuristics and other RL-based methods.
As the demand for multi-chip products with high capacity and small size increases, semiconductor packaging facilities have been faced with complicated constraints such as re-entrant flows, sequence dependent setups, and alternative routes, which leads to difficulties in scheduling semiconductor manufacturing operations. Furthermore, due to the frequent variations in the relative importance between objectives as well as the variabilities in initial setup status, available machines, and production requirements, practitioners are obliged to obtain a schedule within a short amount of computation time. In this paper, we propose a novel two-phase framework that aims to quickly produce a schedule of semiconductor packaging facilities by using case-based reasoning for minimizing the weighted sum of machine loss time and waiting time of jobs. Specifically, in the case generation phase, a case database is constructed by solving case scheduling problems using an existing solver. The case reasoning phase is responsible for repairing operation type sequences in the cases to produce a schedule for an unseen scheduling problem whose production requirements, available machines, initial setup status, and weight between performance measures are different from those of cases. The extensive experimental results demonstrated that the proposed approach requires a short computation time similar to the rule-based methods while maintaining the quality of the schedules comparable to that of the existing metaheuristics.
23 November 2025 in Expert Systems with Applications