4 March 2022 in Journal of Advances in Modeling Earth Systems
Data assimilation (DA) is integrated with machine learning in order to perform entirely data‐driven online state estimation. To achieve this, recurrent neural networks (RNNs) are implemented as pretrained surrogate models to replace key components of the DA cycle in numerical weather prediction (NWP), including the conventional numerical forecast model, the forecast error covariance matrix, and the tangent linear and adjoint models. It is shown how these RNNs can be initialized using DA methods to directly update the hidden/reservoir state with observations of the target system. The results indicate that these techniques can be applied to estimate the state of a system for the repeated initialization of short‐term forecasts, even in the absence of a traditional numerical forecast model. Further, it is demonstrated how these integrated RNN‐DA methods can scale to higher dimensions by applying domain localization and parallelization, providing a path for practical applications in NWP.
22 April 2022 in Earth and Space Science
In most data assimilation and numerical weather prediction systems, the Gaussian assumption is prevalent for the behavior of the random variables/errors that are involved. At the Cooperative Institute for Research in the Atmosphere theory has been developed for different forms of variational data assimilation schemes that enables the Gaussian assumption to be relaxed. For certain variable types, a lognormally distributed random variable can be combined in a mixed Gaussian‐lognormal distribution to better capture the interactions of the errors of different distributions. However, assuming that a distribution can change in time, then developing techniques to know when to switch between different versions of the data assimilation schemes becomes very important. By dynamically changing the formulation of the data assimilation system we are able to assimilate observations in a way that reflects the flow‐dependent variability of their distribution.In this paper, we present results with a machine learning technique (the support vector machine) to switch between data assimilation methods based on the detection of a change in the Lorenz 1963 model's z component's probability distribution. Given the machine learning technique's detection/prediction of a change in the distribution, then either a Gaussian or a mixed Gaussian‐lognormal 3DVar based cost function is used to minimize the errors in this period of time. It is shown that switching from a Gaussian 3DVar to a lognormal 3DVar at lognormally distributed parts of the attractor improves the data assimilation analysis error compared to using one distribution type for the entire attractor.
1 September 2022 in Monthly Weather Review
Four-dimensional variational (4D-Var) data assimilation (DA) is developed for a coupled atmosphere–ocean quasigeostrophic application. Complications arise in coupled data assimilation (CDA) systems due to the presence of multiple spatiotemporal scales. Various formulations of the background error covariance matrix (), using different localization strategies, are explored to evaluate their impact on 4D-Var performance in a CDA setting. 4D-Var requires access to tangent linear and adjoint models (TLM/AM) to propagate information about the misfit between the forecast and observations within an optimization window. In practice, particularly for coupled models, the TLM and adjoint are often difficult to produce, and for some models are nonexistent in analytic form. Accordingly, a statistical data-driven alternative is also employed and evaluated to determine its feasibility for a 4D-Var CDA system. Using experiments conducted with a coupled atmosphere–ocean quasigeostrophic model, it is found that ensemble generation of flow-dependent error covariance statistics can increase the accuracy of 4D-Var CDA. When observing all variables, the hybrid climatological/flow-dependent constructions outperform either independently. The use of a hybrid matrix combined with a rapid updating ensemble transform Kalman filter (RU-ETKF) using either strongly or weakly CDA resulted in lower overall RMSE. The ocean component achieved its lowest RMSE when using a fully flow-dependent matrix generated using 4D-ETKF and using weakly CDA. These results show the importance of time scales and analysis update frequencies. The use of a statistically derived TLM/AM generated from the ETKF ensemble perturbations produces results similar to cases using the analytical coupled TLM/AM in 4D-Var.
28 September 2021 in ESS Open Archive
HAHenry Don Isaac Abarbanel
Henry Don Isaac Abarbanel
+1
Data assimilation (DA) is integrated with machine learning in order to perform entirely data-driven online state estimation. To achieve this, recurrent neural networks (RNNs) are implemented as surrogate models to replace key components of the DA cycle in numerical weather prediction (NWP), including the conventional numerical forecast model, the forecast error covariance matrix, and the tangent linear and adjoint models. It is shown how these RNNs can be initialized using DA methods to directly update the hidden/reservoir state with observations of the target system. The results indicate that these techniques can be applied to estimate the state of a system for the repeated initialization of short-term forecasts, even in the absence of a traditional numerical forecast model. Further, it is demonstrated how these integrated RNN-DA methods can scale to higher dimensions by applying domain localization and parallelization, providing a path for practical applications in NWP.