Scientific machine learning (SciML) models are transforming many scientific disciplines. However, the development of good modeling practices to increase the trustworthiness of SciML has lagged behind its application, limiting its potential impact. The goal of this paper is to start a discussion on establishing consensus-based good practices for predictive SciML. We identify key challenges in applying existing computational science and engineering guidelines, such as verification and validation protocols, and provide recommendations to address these challenges. Our discussion focuses on predictive SciML, which uses machine learning models to learn, improve, and accelerate numerical simulations of physical systems. While centered on predictive applications, our 16 recommendations aim to help researchers conduct and document their modeling processes rigorously across all SciML domains.

Aerodynamic design optimization improves aircraft performance and reduces emissions, but conventional approaches iterate on high-fidelity simulations, creating a computational bottleneck. Surrogate-based design optimization can address this bottleneck, but training surrogate models is costly and surrogate-based design requires optimization iterations, which can be prohibitive for resource-limited applications. To eliminate the need for online optimization, we propose a new aerodynamic design architecture—inverse mapping. This architecture uses a surrogate to directly predict optimal airfoil shapes from design requirements, including constraints and flight conditions. The inverse mapping poses two challenges: high complexity and the cost of generating training samples through simulation-based optimizations. We address these challenges by exploiting a low-rank space via singular value decomposition to reduce the mapping complexity and by using multifidelity surrogate modeling to reduce the computational cost. We demonstrate the inverse-mapping architecture on airfoil design in transonic viscous flows with fixed constraints setup. A single-fidelity surrogate trained on the full set of design variables achieves 95% accuracy with 140 high-fidelity training samples, and training on the reduced-order variables lowers this to 120 samples. Combining the reduced-order representation with a multifidelity surrogate—using both high- and low-fidelity training data—further reduces the requirement to 80 high-fidelity and 100 low-fidelity samples, equivalent to 85 high-fidelity samples given the 20-to-1 cost ratio, a 29.2% reduction in computational cost. Robustness tests confirm consistently strong performance. In addition, a 95% accuracy in the airfoil shape variables yields 0.5% error in airfoil coordinates and 0.15% error in drag coefficients. The inverse-mapping architecture enables real-time design and generalizes to other engineering applications.

We consider minimizing finite-sum objective functions via Hessian-averaging based subsampled Newton methods. These methods allow for gradient inexactness and have fixed per-iteration Hessian approximation costs. The recent work (Na et al. 2023) demonstrated that Hessian averaging can be utilized to achieve fast $$\mathcal {O}\left( \sqrt{\tfrac{\log k}{k}}\right) $$ O log k k local superlinear convergence for strongly convex functions in high probability, while maintaining fixed per-iteration Hessian costs. These methods, however, require gradient exactness and strong convexity, which poses challenges for their practical implementation. To address this concern we consider Hessian-averaged methods that allow gradient inexactness via norm condition based adaptive-sampling strategies. Furthermore, to better control the error in the subsampled Hessian approximations, we utilize Hessian averaging with deterministic cyclic sampling techniques instead of random sampling, which leads to fast local superlinear convergence. We develop a comprehensive convergence theory, including global linear and sublinear convergence rates for strongly convex and nonconvex functions, respectively. Additionally, we establish an improved local superlinear convergence rate of $$\mathcal {O}\left( \tfrac{1}{k}\right) $$ O 1 k . Our analysis introduces novel techniques that differ from previous probabilistic approaches. We investigate the performance of these methods on logistic regression problems, demonstrating significant improvements in convergence over similar Hessian-averaging methods that utilize stochastic sampling.