The digitalization of manufacturing processes offers great potential in quality control, traceability, and the planning and setup of production. In this regard, process simulation is a well-known technology and a key step in the design of manufacturing processes. However, process simulations are computationally and time-expensive, typically beyond the manufacturing-cycle time, severely limiting their usefulness in real-time process control. Machine Learning-based surrogate models can overcome these drawbacks, and offer the possibility to achieve a soft real-time response, which can be potentially developed into full close-loop manufacturing systems, at a computational cost that can be realistically implemented in an industrial setting. This paper explores the novel concept of using a surrogate model to analyze the case of the press hardening of a steel sheet of 22MnB5. This hot sheet metal forming process involves a crucial heat treatment step, directly related to the final part quality. Given its common use in high-responsibility automobile parts, this process is an interesting candidate for digitalization in order to ensure production quality and traceability. A comparison of different data and model training strategies is presented. Finite element simulations for a transient heat transfer analysis are performed with ABAQUS software and they are used for the training data generation to effectively implement a ML-based surrogate model capable of predicting key process outputs for entire batch productions. The resulting final surrogate predicts the behavior and evolution of the most important temperature variables of the process in a wide range of scenarios, with a mean absolute error around 3 °C, but reducing the time four orders of magnitude with respect to the simulations. Moreover, the methodology presented is not only relevant for manufacturing purposes, but can be a technology enabler for advanced systems, such as digital twins and autonomous process control.

This paper explores the application of offline reinforcement learning in batch manufacturing, with a specific focus on press hardening processes. Offline reinforcement learning presents a viable alternative to traditional control and reinforcement learning methods, which often rely on impractical real-world interactions or complex simulations and iterative adjustments to bridge the gap between simulated and real-world environments. We demonstrate how offline reinforcement learning can improve control policies by leveraging existing data, thereby streamlining the training pipeline and reducing reliance on high-fidelity simulators. Our study evaluates the impact of varying data exploration rates by creating five datasets with exploration rates ranging from ε=0 to ε=0.8. Using the conservative Q-learning algorithm, we train and assess policies against both a dynamic baseline and a static industry-standard policy. The results indicate that while offline reinforcement learning effectively refines behavior policies and enhances supervised learning methods, its effectiveness is heavily dependent on the quality and exploratory nature of the initial behavior policy.

Hot stamping is a hot metal forming technology increasingly in demand that produces ultra-high strength parts with complex shapes. A major concern in these systems is how to shorten production times to improve production Key Performance Indicators. In this work, we present a Reinforcement Learning approach that can obtain an optimal behavior strategy for dynamically managing the cycle time in hot stamping to optimize manufacturing production while maintaining the quality of the final product. Results are compared with the business-as-usual cycle time control approach and the optimal solution obtained by the execution of a dynamic programming algorithm. Reinforcement Learning control outperforms the business-as-usual behavior by reducing the cycle time and the total batch time in non-stable temperature phases.

A holistic Water-Energy-Food-Ecosystems (WEFE) nexus approach requires advanced decision-support systems capable of navigating complex interdependencies and optimizing trade-offs among multiple, often conflicting objectives. Moreover, the WEFE Nexus aims not only to promote efficiency in cross-sectoral interactions but also to ensure fairness by distributing resources in a fair and just manner. However, decision-making is further complicated by uncertainty in stakeholder preferences, as diverse actors often hold conflicting priorities and ambiguous visions of what constitutes an acceptable outcome. To address these challenges, this study presents a fairness-aware Multi-Objective Reinforcement Learning (MORL) framework based on Pareto Q-Learning, designed to support policy-making in high-dimensional objective spaces under preference uncertainty. The framework integrates a hybrid utility function combining efficiency and fairness with the real-time response of a learning approach, where a combination of a weighted prioritization mechanism with a targeted standard deviation regularization term enables dynamic balancing of competing objectives while preserving Pareto-optimality. We apply our methodology to the Inkomati-Usuthu river basin, a transboundary region shared by South Africa, Eswatini, and Mozambique. This basin plays a critical role in regional energy production, agricultural development, and water security, while also supporting ecologically sensitive ecosystems. Building on system dynamics modelling, the framework generates AI-driven recommendations for policy packages tailored to stakeholder-defined preferences, supports efficient many-objective optimization, and enables transparent identification of cross-sectoral impacts, synergies, and trade-offs across the WEFE Nexus. Our findings contribute to the growing body of research on AI-enabled governance, demonstrating how fairness-aware MORL can enhance real-time many-objective policy optimization, foster cross-sectoral coordination, improve resilience, and strengthen transboundary basin governance.