The term NeuralODE describes the structural combination of an Artificial Neural Network (ANN) and a numerical solver for Ordinary Differential Equations (ODE), the former acts as the right-hand side of the ODE to be solved. This concept was further extended by a black-box model in the form of a Functional Mock-up Unit (FMU) to obtain a subclass of NeuralODEs, named NeuralFMUs. The resulting structure features the advantages of the first-principle and data-driven modeling approaches in one single simulation model: a higher prediction accuracy compared to conventional First-Principle Models (FPMs) and also a lower training effort compared to purely data-driven models. We present an intuitive workflow to set up and use NeuralFMUs, enabling the encapsulation and reuse of existing conventional models exported from common modeling tools. Moreover, we exemplify this concept by deploying a NeuralFMU for a consumption simulation based on a Vehicle Longitudinal Dynamics Model (VLDM), which is a typical use case in the automotive industry. Related challenges that are often neglected in scientific use cases, such as real measurements (e.g., noise), an unknown system state or high-frequency discontinuities, are handled in this contribution. To build a hybrid model with a higher prediction quality than the original FPM, we briefly highlight two open-source libraries: FMI.jl, which allows for the import of FMUs into the Julia programming language, as well as the library FMIFlux.jl, which enables the integration of FMUs into neural network topologies to obtain a NeuralFMU.

The Functional Mock-up Interface (FMI) is the standard forexchanging industrial simulation models in a variety ofdifferent applications. Although sensitivity analysis forcontinuously differentiable systems is directly supportedby the standard, for systems with state discontinuities, itis only possible to determine correct sensitivities to alimited extent. In this position paper, we investigate howsensitivity analysis for discontinuous Functional Mock-upUnits (FMUs), i.e. including state and time events, worksin theory and which additional steps are required to obtaincorrect results in practice. We further investigate thatthese steps are unnecessarily computationally intensivefrom a mathematical point of view, but cannot beimplemented in a more efficient way under the currentrestrictions of the standard. We therefore make a concreteproposal for the new layered standard sensitivity analysis(LS-SA) that remedies the current deficits of FMI in thesensitivity analysis of discontinuous systems. In this way,LS-SA opensFMI towards a variety of next-level applications —including (scientific) machine learning and optimal control— by providing fully differentiable FMUs under highcomputational performance.

Porting Neural Ordinary Differential Equations (NeuralODEs), the combination of an artificial neural network and an ODE solver, to real engineering applications is still a challenging venture. However, we will show that Neural Functional Mock-up Units (NeuralFMUs), an evolved subgroup of NeuralODEs that contain Functional Mock-up Units (FMUs), are able to cope with these challenges. This paper briefly introduces to the topics NeuralODE and NeuralFMU and describes the procedure and considerations to apply this technique to a real engineering use case. Further, different workflows to apply NeuralFMUs dependent on tool capabilities and use case requirements are discussed. The presented method is illustrated with the creation of a Hybrid Twin of an hydraulic excavator arm, which has various challenges such as discontinuity, nonlinearity, oscillations and characteristic maps. Finally we will show, that the created Hybrid Twin, on basis of measurement data from a real system, gives more accurate results compared to a conventional simulation model based on first principles.

Using vanilla NeuralODEs to model large and/or complex systems often fails due two reasons: Stability and convergence. NeuralODEs are capable of describing stable as well as instable dynamic systems. Selecting an appropriate numerical solver is not trivial, because NeuralODE properties change during training. If the NeuralODE becomes more stiff, a suboptimal solver may need to perform very small solver steps, which significantly slows down the training process. If the NeuralODE becomes to instable, the numerical solver might not be able to solve it at all, which causes the training process to terminate. Often, this is tackled by choosing a computational expensive solver that is robust to instable and stiff ODEs, but at the cost of a significantly decreased training performance. Our method on the other hand, allows to enforce ODE properties that fit a specific solver or application-related boundary conditions. Concerning the convergence behavior, NeuralODEs often tend to run into local minima, especially if the system to be learned is highly dynamic and/or oscillating over multiple periods. Because of the vanishing gradient at a local minimum, the NeuralODE is often not capable of leaving it and converge to the right solution. We present a technique to add knowledge of ODE properties based on eigenvalues - like (partly) stability, oscillation capability, frequency, damping and/or stiffness - to the training objective of a NeuralODE. We exemplify our method at a linear as well as a nonlinear system model and show, that the presented training process is far more robust against local minima, instabilities and sparse data samples and improves training convergence and performance.

Testing real hardware and simulation models in combinationin a software- or hardware-in-the-loop set-up ischallenging. One of the key factors is the high demand foraccuracy in the simulation model. If classical modelingbased on physical principles is not sufficient to reach thedesired level of accuracy, hybrid modeling, the combinationof physical simulation models and machine learning can beapplied. In this publication, we train a hybrid model for acontrolled electric motor within the electro-hydraulicbraking system of a car under the conditions andrestrictions of a real engineering application in thefield. We apply state-of-the-art modeling patterns forthis, and further extend them with application specificmethodological optimizations. Finally, we investigate andshow the quantitative and qualitative advantages of theproposed approach for this specific application, resultingin a gain in accuracy by multiple factors.