In order to not only produce an open-die forged part with the desired final geometry but to also maintain economic production, precise process planning is necessary. However, due to the incremental forming of the billet, often with several hundred strokes, the process design is arbitrarily complicated and, even today, often only based on experience or simple mathematical models describing the geometry development. Hence, in this paper, fast process models were merged with a double deep Q-learning algorithm to enable a pass schedule design including multi-objective optimization. The presented implementation of a double deep Q-learning algorithm was successfully trained on an industrial-scale forging process and converged stably against high reward values. The generated pass schedules reliably produced the desired final ingot geometry, utilized the available press force well without exceeding plant limits, and, at the same time, minimized the number of passes. Finally, a forging experiment was performed at the institute of metal forming to validate the generated results. Overall, a proof of concept for the pass schedule design in open-die forging via double deep Q-learning was achieved which opens various starting points for future work.

Metal forming processes like open-die forging or hot rolling are well-established for the production of key components in various industries. Nevertheless, the control of the final microstructure and hence mechanical properties is not yet common. To achieve this, the authors propose and discuss a control concept based on reinforcement learning, fast process models (FPM) and an “operator in the loop” approach. The concept is explained and tested using deviating initial ingot temperatures as idealized process disruptions. RL algorithms are trained for both processes and transferred into the controllers that are connected to a simulative environment based on FPM. Within this framework, the online adaption is possible in ∼2 s in rolling and 4–6 s in forging. This highlights the concepts suitability to be used for property control in hot metal forming.

In this chapter, the focus lies on a predictive description of the material response to the thermomechanical loads within different process steps by means of physical and data-driven models. The modeling approaches are demonstrated in examples of innovative production technologies for components of a drive chain: Fine blanking of parts; powder metallurgical (PM) production of gears; open-die forging and machining of drive shafts. In fine blanking, material, process, and quality data are acquired to model interactions between process and material with data-driven methods. Interpretable machine learning is utilized to non-destructively characterize the initial material state, enabling an optimization of process parameters for a given material state in the long-term. The PM process chain of the gear includes sintering, pressing, surface densification, case hardening, and finishing by grinding. Several modeling and characterization approaches are applied to quantitatively describe the microstructure evolutions in terms of porosity during sintering, density profile after cold rolling, hardness and residual stresses after heat treating and grinding and the tooth root load bearing capacity. In the example of the open-die forging, a knowledge-based approach is developed to support the decision-making process regarding the choice of the proper material and optimized pass schedules. Considering the microstructure of the forged shaft, the elastoplastic material behavior is described by a dislocation-based, multiscale modeling approach. On this basis, process simulations could be carried out to predict the process forces, chip form, residual stresses, and the tool life among other output data.

The open-die forging process can produce large workpieces with excellent material properties that can be used for heavy-duty applications like turbine shafts. The mechanical properties result from the microstructure, which in turn directly results from the process route. Since in open-die forging processes commonly hundreds of unique forming operations are carried out, numerous process routes lead to the same final geometry but produce different microstructures. This is why the prior design of an optimal pass schedule is essential to ensure good mechanical properties of open-die forgings. In the past reinforcement learning (RL) was already used to design optimized pass schedules for open-die forging that achieve the desired geometry, utilize the available press force, and reduce the number of passes. Furthermore, the design of a single pass schedule only took a few seconds which creates opportunities for the use of RL in control systems e.g. for the microstructure. This is why in this publication an existing RL algorithm is extended so that microstructure can be included in the optimization. Within this process, a microstructure model is integrated into the RL algorithm and the reward function (defines the goal of the training process) was extended in two steps to also rate the achieved average grain size continuously dependent on the temperature of the workpiece. In addition, the RL implementation was changed to ensure the production of the desired final geometry leading to a decrease in the complexity of the optimization problem. Thus, both an improvement of the designed pass schedules and a significant reduction of the training time of the RL algorithm was achieved.

In this chapter, the focus lies on a predictive description of the material response to the thermomechanical loads within different process steps by means of physical and data-driven models. The modeling approaches are demonstrated in examples of innovative production technologies for components of a drive chain: Fine blanking of parts; powder metallurgical (PM) production of gears; open-die forging and machining of drive shafts. In fine blanking, material, process, and quality data are acquired to model interactions between process and material with data-driven methods. Interpretable machine learning is utilized to non-destructively characterize the initial material state, enabling an optimization of process parameters for a given material state in the long-term. The PM process chain of the gear includes sintering, pressing, surface densification, case hardening, and finishing by grinding. Several modeling and characterization approaches are applied to quantitatively describe the microstructure evolutions in terms of porosity during sintering, density profile after cold rolling, hardness and residual stresses after heat treating and grinding and the tooth root load bearing capacity. In the example of the open-die forging, a knowledge-based approach is developed to support the decision-making process regarding the choice of the proper material and optimized pass schedules. Considering the microstructure of the forged shaft, the elastoplastic material behavior is described by a dislocation-based, multiscale modeling approach. On this basis, process simulations could be carried out to predict the process forces, chip form, residual stresses, and the tool life among other output data.