We introduce a data-driven forecasting method for high-dimensional chaotic systems using long short-term memory (LSTM) recurrent neural networks. The proposed LSTM neural networks perform inference of high-dimensional dynamical systems in their reduced order space and are shown to be an effective set of nonlinear approximators of their attractor. We demonstrate the forecasting performance of the LSTM and compare it with Gaussian processes (GPs) in time series obtained from the Lorenz 96 system, the Kuramoto–Sivashinsky equation and a prototype climate model. The LSTM networks outperform the GPs in short-term forecasting accuracy in all applications considered. A hybrid architecture, extending the LSTM with a mean stochastic model (MSM–LSTM), is proposed to ensure convergence to the invariant measure. This novel hybrid method is fully data-driven and extends the forecasting capabilities of LSTM networks.

The prediction of extreme events, from avalanches and droughts to tsunamis and epidemics, depends on the formulation and analysis of relevant, complex dynamical systems. Such dynamical systems are characterized by high intrinsic dimensionality with extreme events having the form of rare transitions that are several standard deviations away from the mean. Such systems are not amenable to classical order-reduction methods through projection of the governing equations due to the large intrinsic dimensionality of the underlying attractor as well as the complexity of the transient events. Alternatively, data-driven techniques aim to quantify the dynamics of specific, critical modes by utilizing data-streams and by expanding the dimensionality of the reduced-order model using delayed coordinates. In turn, these methods have major limitations in regions of the phase space with sparse data, which is the case for extreme events. In this work, we develop a novel hybrid framework that complements an imperfect reduced order model, with data-streams that are integrated though a recurrent neural network (RNN) architecture. The reduced order model has the form of projected equations into a low-dimensional subspace that still contains important dynamical information about the system and it is expanded by a long short-term memory (LSTM) regularization. The LSTM-RNN is trained by analyzing the mismatch between the imperfect model and the data-streams, projected to the reduced-order space. The data-driven model assists the imperfect model in regions where data is available, while for locations where data is sparse the imperfect model still provides a baseline for the prediction of the system state. We assess the developed framework on two challenging prototype systems exhibiting extreme events. We show that the blended approach has improved performance compared with methods that use either data streams or the imperfect model alone. Notably the improvement is more significant in regions associated with extreme events, where data is sparse.

Extreme events, such as rogue waves, earthquakes, and stock market crashes, occur spontaneously in many dynamical systems. Because of their usually adverse consequences, quantification, prediction, and mitigation of extreme events are highly desirable. Here, we review several aspects of extreme events in phenomena described by high-dimensional, chaotic dynamical systems. We especially focus on two pressing aspects of the problem: (i) mechanisms underlying the formation of extreme events and (ii) real-time prediction of extreme events. For each aspect, we explore methods relying on models, data, or both. We discuss the strengths and limitations of each approach as well as possible future research directions.

A variational framework for the analysis and data-driven prediction of extreme events is developed.

Significance: Understanding the statistics of extreme events in dynamical systems of high complexity is of vital importance for reliability assessment and design. We formulate a method to pick samples optimally so that we have rapid convergence of the full statistics of a quantity of interest, including the tails that describe extreme events. This is important for large-scale problems in science and engineering, where we desire to predict the statistics of relevant quantities but can only afford a limited number of simulations or experiments due to their very expensive cost. We demonstrate our approach in a hydromechanical system with millions of degrees of freedom, where only 10–20 carefully selected samples can lead to accurate approximation of the extreme event statistics.