This proof-of-concept study provides a novel method for robust-stable scheduling in dynamic flow shops based on deep reinforcement learning (DRL) implemented with OpenAI frameworks. In realistic manufacturing environments, dynamic events endanger baseline schedules, which can require a cost intensive re-scheduling. Extensive research has been done on methods for generating proactive baseline schedules to absorb uncertainties in advance and in balancing the competing metrics of robustness and stability. Recent studies presented exact methods and heuristics based on Monte Carlo experiments (MCE), both of which are very computationally intensive. Furthermore, approaches based on surrogate measures were proposed, which do not explicitly consider uncertainties and robustness metrics. Surprisingly, DRL has not yet been scientifically investigated for generating robust-stable schedules in the proactive stage of production planning. The contribution of this article is a proposal on how DRL can be applied to manipulate operation slack times by stretching or compressing plan durations. The method is demonstrated using different flow shop instances with uncertain processing times, stochastic machine failures and uncertain repair times. Through a computational study, we found that DRL agents achieve about 98% result quality but only take about 2% of the time compared to traditional metaheuristics. This is a promising advantage for the use in real-time environments and supports the idea of improving proactive scheduling methods with machine learning based techniques.

The following interdisciplinary article presents a memetic algorithm with applying deep reinforcement learning (DRL) for solving practically oriented dual resource constrained flexible job shop scheduling problems (DRC-FJSSP). From research projects in industry, we recognize the need to consider flexible machines, flexible human workers, worker capabilities, setup and processing operations, material arrival times, complex job paths with parallel tasks for bill of material (BOM) manufacturing, sequence-dependent setup times and (partially) automated tasks in human-machine-collaboration. In recent years, there has been extensive research on metaheuristics and DRL techniques but focused on simple scheduling environments. However, there are few approaches combining metaheuristics and DRL to generate schedules more reliably and efficiently. In this paper, we first formulate a DRC-FJSSP to map complex industry requirements beyond traditional job shop models. Then we propose a scheduling framework integrating a discrete event simulation (DES) for schedule evaluation, considering parallel computing and multicriteria optimization. Here, a memetic algorithm is enriched with DRL to improve sequencing and assignment decisions. Through numerical experiments with real-world production data, we confirm that the framework generates feasible schedules efficiently and reliably for a balanced optimization of makespan (MS) and total tardiness (TT). Utilizing DRL instead of random metaheuristic operations leads to better results in fewer algorithm iterations and outperforms traditional approaches in such complex environments.

Solving production scheduling problems is a difficult and indispensable task for manufacturers with a push-oriented planning approach. In this study, we tackle a novel production scheduling problem from a household appliance production at the company Miele & Cie. KG, namely a two-stage permutation flow shop scheduling problem (PFSSP) with a finite buffer and sequence-dependent setup efforts. The objective is to minimize idle times and setup efforts in lexicographic order. In extensive and realistic data, the identification of exact solutions is not possible due to the combinatorial complexity. Therefore, we developed a reinforcement learning (RL) approach based on the Proximal Policy Optimization (PPO) algorithm that integrates domain knowledge through reward shaping, action masking, and curriculum learning to solve this PFSSP. Benchmarking of our approach with a state-of-the-art genetic algorithm (GA) showed significant superiority. Our work thus provides a successful example of the applicability of RL in real-world production planning, demonstrating not only its practical utility but also showing the technical and methodological integration of the agent with a discrete event simulation (DES). We also conducted experiments to investigate the impact of individual algorithmic elements and a hyperparameter of the reward function on the overall solution.

This feasibility study utilized regression models to predict makespan robustness in dynamic production processes with uncertain processing times. Previous methods for robustness determination were computationally intensive (Monte Carlo experiments) or inaccurate (surrogate measures). However, calculating robustness efficiently is crucial for field-synchronous scheduling techniques. Regression models with multiple input features considering uncertain processing times on the critical path outperform traditional surrogate measures. Well-trained regression models internalize the behavior of a dynamic simulation and can quickly predict accurate robustness (correlation: $r>0.98$). The proposed method was successfully applied to a permutation flow shop scheduling problem, balancing makespan and robustness. Integrating regression models into a metaheuristic model, schedules could be generated that have a similar quality to using Monte Carlo experiments. These results suggest that employing machine learning techniques for robustness prediction could be a promising and efficient alternative to traditional approaches. This work is an addition to our previous extensive study about creating robust stable schedules based on deep reinforcement learning and is part of the applied research project, Predictive Scheduling.

Angesichts des zunehmenden Fachkräftemangels sind Gesundheit und Zufriedenheit der Beschäftigten von immer größerer Bedeutung für produzierende Unternehmen. Insbesondere bei variantenreichen Produkten müssen Werker und Werkerinnen komplexe Aufgaben erledigen. Diese Variantenvielfalt ist auch für die Produktionsfeinplanung eine Herausforderung. Die vorgestellte Heuristik hilft in einem realen Anwendungsfall Belastungsspitzen an getakteten Linien zu reduzieren und verbessert so die betriebliche Gesundheit. Given a shortage of skilled workers, employee health and satisfaction are of increasing importance to manufacturing companies. Particularly in the case of products with many variants, workers have to perform complex tasks. This diversity of variants is also a challenge for production planning. New planning paradigms help to reduce peaks of stress on flow production and thus contribute to occupational health.