The last decade has seen the development and application of data-driven methods taking off in nuclear engineering research, aiming to improve the safety and reliability of nuclear power. This work focuses on developing a reinforcement learning-based control sequence optimization framework for advanced nuclear systems, which not only aims to enhance flexible operations, promoting the economics of advanced nuclear technology, but also prioritizing safety during normal operation. At its core, the framework allows the sequence of operational actions to be learned and optimized by an agent to facilitate smooth transitions between the modes of operations (i.e., load-following), while ensuring that all safety significant system parameters remain within their respective limits. To generate dynamic system responses, facilitate control strategy development, and demonstrate the effectiveness of the framework, a simulation environment of a pebble-bed high-temperature gas-cooled reactor was utilized. The soft actor-critic algorithm was adopted to train a reinforcement learning agent, which can generate control sequences to maneuver plant power output in the range between 100% and 50% of the nameplate power through sufficient training. It was shown in the performance validation that the agent successfully generated control actions that maintained electrical output within a tight tolerance of 0.5% from the demand while satisfying all safety constraints. During the mode transition, the agent can maintain the reactor outlet temperature within $\pm 1.5$ °C and steam pressure within 0.1 MPa of their setpoints, respectively, by dynamically adjusting control rod positions, control valve openings, and pump speeds. The results demonstrate the effectiveness of the optimization framework and the feasibility of reinforcement learning in designing control strategies for advanced reactor systems.

In the pursuit of the ability to perform multiphysics simulations of the Advanced Test Reactor, it is crucial to have a fast and highly accurate deterministic model. To achieve this, a contemporary two-step method is employed. The first step involves generating homogenized cross sections using OpenMC, a cutting-edge Monte Carlo neutron transport code. OpenMC offers excellent modular capabilities, allowing for easy component integration and flexibility in incorporating new designs into the model. The second step involves deterministic transport calculations, which are performed using Griffin, a reactor multiphysics application based on the Multiphysics Object-Oriented Simulation Environment. To ensure the accurate spatial resolution and assignment of material cross sections, a Cubit-generated mesh for the Advanced Test Reactor is utilized as an intermediate step between the OpenMC and Griffin models; Griffin utilizes the mesh for its finite element solution, while OpenMC material IDs are written to the mesh file to be used in Griffin material assignments. Additionally, a Python-based script converts the cross sections generated by OpenMC into the ISOXML format required by Griffin. Preliminary comparisons indicate good agreement between the neutron multiplication factors obtained from the standalone OpenMC model and the Griffin model, with differences of less than 50 pcm in themore » two-dimensional geometry configuration. However, in three-dimensional calculations, an unacceptably large error is found in the Griffin solution. Future work is planned to resolve this discrepancy. « less

As energy demand continues to grow globally, cost competitiveness and safety remain key challenges in the nuclear energy sector. The Modularly Implemented Design Assistance Suite (MIDAS) was developed to support and enhance nuclear engineering design through optimization algorithms that can be easily adapted to any problem. MIDAS is written in Python 3, and uses object-oriented programming to create a modular architecture that allows users to easily expand its capabilities and incorporate new optimization algorithms and problems. MIDAS employs a standardized YAML input file, allowing users to easily specify design problems, change between code interfaces, and apply various algorithms. The latest version of MIDAS contains: Genetic Algorithm, Simulated Annealing, Bayesian Optimization, and Parallel Simulated Annealing Algorithms, each containing a number of hyperparameters allowing users to change optimization dynamics. The effectiveness of the algorithms was demonstrated in a verification problem, as well as a single cycle loading pattern, equilibrium cycle loading pattern, and lattice physics optimizations. In all demonstrations, the algorithms efficiently searched the design spaces and found optimized solutions to the given problems. Due to its modular architecture, MIDAS is well-positioned as a solution to nuclear engineering optimization problems, and future developments including applications and algorithms will be easily incorporated into the framework.